AppDynamics Application Performance Monitoring for VMware Tanzu Workflow

This topic describes the steps required to instrument VMWare Tanzu applications with Splunk AppDynamics agents using the marketplace service published by the Splunk AppDynamics APM tile. For all but the Java buildpack, the multi-buildpack approach is used, which requires the Splunk AppDynamics extension buildpack published by the tile, in addition to the language-specific buildpack. The Java buildpack has the Splunk AppDynamics agent support is directly integrated and does not require the use of the Splunk AppDynamics buildpack.

Before You Start

1. Create a Pivotal Network account if you do not have one.
2. Install the Cloud Foundry Command Line Interface (cf CLI). See Installing the cf CLI.

This document contains references to Cloud Foundry™ documentation. Splunk AppDynamics does not own any rights and assumes no responsibility for the accuracy or completeness of such third-party documentation.

Create a Splunk AppDynamics Service Instance

1. Open a cf CLI or command-prompt window.

2. Identify your Pivotal API endpoint and version. Enter the following command:
   $ cf api 
   The following output appears:

   api endpoint: https<scan>:</scan>//api.my.pivotal.io
   api version: 2.103.0

   CODE

3. Log in to the Pivotal endpoint: cf login

4. Enter your email address and Pivotal password when prompted.

5. Verify the appdynamics service is present in the cf environment.

   $ cf marketplace -s appdynamics
   service plan                   description                    free or paid
   pcf-appd                       pcf-appd                       free
   pcf-appd-test                  pcf-appd-test                  free

   CODE

6. Create a service instance of the appdynamics service and a plan of your choice. $ cf create-service appdynamics pcf-appd appd

7. Follow the workflow for the corresponding language of the application mentioned below.

Splunk AppDynamics Instrumentation Workflows

Sample Applications

You can find sample applications demonstrating the multi-buildpack approach in this GitHub repository.

.NET Framework Applications (Multi-Buildpack)

.Net Framework applications are pushed using the multi-buildpack approach where the appdbuildpack buildpack is used in tandem with the standard hwc_buildpack buildpack.

1. Edit the following sections of your application’s manifest.yml.

2. Include appdbuildpack in the buildpacks section.

3. Set the application environment variable APPD_AGENT to dotnet.

4. Set the app and tier environment variables.

5. Bind the application to the Splunk AppDynamics service instance in the services section of the application manifest.

   ---
   applications:
   - name: cf-net
     memory: 2G
     buildpacks:
   	- appdbuildpack
   	- hwc_buildpack
     stack: windows2016
     env:
   	APPD_AGENT: dotnet
   	APPDYNAMICS_AGENT_APPLICATION_NAME: my-cf-net-app
   	APPDYNAMICS_AGENT_TIER_NAME: cf-net-app
     services:
   	- appd

   CODE

6. Push the application using the cf push command.
   

   $ cf push

   CODE

   
   

Advanced Configuration

appdbuildpack provides APPD_AGENT_HTTP_URL and APPD_CONF_HTTP_URL environment variables to customize agent binary downloads and extend agents advanced configuration respectively. See Multi-Buildpack documentation for details.

.NET Core for Linux Applications (Multi-Buildpack)

.Net Core applications are pushed using the multi-buildpack approach where the appdbuildpack buildpack is used in tandem with the standard dotnet_core_buildpack buildpack.

1. Edit the following sections of your application’s manifest.yml.

2. Include appdbuildpack in the buildpacks section.

3. Set the application environment variable APPD_AGENT to dotnet-linux.

4. Set the app and tier environment variables.

5. Set the application environment variable APPD_AGENT_HTTP_URL to a custom http url where the .NET Agent for Linux is hosted. This is necessary because the agent is not currently available in NuGet. See Cisco AppDynamics Extension Buildpack for more information.

6. Set the APPD_CONF_HTTP_URL to a custom url where the the agent configuration file AppDynamicsConfig.json is hosted in the location
   ${APPD_CONF_HTTP_URL}/dotnet-linux/AppDynamicsConfig.json with the following contents:

   {
   	"feature": [
   		"FULL_AGENT"
   	],
   	"log": [
   		{
   		"outputtype": "console"
   		}
   	]
   }

   CODE

7. Bind the application to the Splunk AppDynamics service instance in the services section of the application manifest.

   ----
   applications:
   - name: cf-net-linux
     memory: 2G
     buildpacks:
   	- appdbuildpack
   	- dotnet_core_buildpack
     env:
   	APPD_AGENT: dotnet-linux
   	APPDYNAMICS_AGENT_APPLICATION_NAME: my-dotnet-linux-app
   	APPDYNAMICS_AGENT_TIER_NAME: cf-net-linux
   	APPD_AGENT_HTTP_URL: http://CUSTOM_REPO/AppDynamics-DotNetCore-linux-x64-VERSION.zip
   	APPD_CONF_HTTP_URL: http://CUSTOM_REPO
     services:
   	- appd

   CODE

8. Push the application using the cf push command.

   $ cf push

   CODE

Advanced Configuration

appdbuildpack provides APPD_AGENT_HTTP_URL and APPD_CONF_HTTP_URL environment variables to customize agent binary downloads and extend agents advanced configuration respectively. Refer to Multi-Buildpack documentation for details.

.NET Core Windows Applications (Multi-Buildpack)

.Net Core applications for Windows are pushed using the multi-buildpack approach where the appdbuildpack buildpack is used in tandem with the standard binary_buildpack buildpack.

1. Edit the following sections of your application’s manifest.yml.

2. Include appdbuildpack in the buildpacks section.

3. Set the application environment variable APPD_AGENT to dotnet-windows.

4. Set the app and tier environment variables.

5. Bind the application to the Splunk AppDynamics service instance in the services section of the application manifest.

   ---
   applications:
   - name: cf-net-windows
     random-route: true
     memory: 512M
     path: app.zip
     command: dotnetHelloWorld.exe
     stack: windows2016
     buildpacks:
   	- appdbuildpack
   	- binary_buildpack
     env:
   	APPD_AGENT: dotnet-windows
   	APPDYNAMICS_AGENT_APPLICATION_NAME: my-dotnet-windows-app
   	APPDYNAMICS_AGENT_TIER_NAME: cf-net-windows
   	DOTNET_CLI_TELEMETRY_OPTOUT: 1
   	DOTNET_SKIP_FIRST_TIME_EXPERIENCE: true
     services:
   	- appd

   CODE

6. Push the application using the cf push command.
   

   cf push

   CODE

Advanced Configuration

appdbuildpack provides APPD_AGENT_HTTP_URL and APPD_CONF_HTTP_URL environment variables to customize agent binary downloads and extend agents advanced configuration respectively. Refer to Multi-Buildpack documentation for details.

Python Applications (Multi-Buildpack)

Python applications are pushed using the multi-buildpack approach where the appdbuildpack buildpack is used in tandem with the standard python_buildpack buildpack.

1. Edit the following sections of your application’s manifest.yml.

2. Include appdbuildpack in the buildpacks section of application manifest.

3. Set the application environment variable APPD_AGENT to python.

4. Set the app and tier environment variables.

5. Bind the application to the Splunk AppDynamics service instance in the services section of the application manifest.

   ---
   applications:
   - name: cf-python
     memory: 500M
     buildpacks:
   	- appdbuildpack
   	- python_buildpack
     env:
   	APPD_AGENT: python
   	APPD_APP_NAME: my-python-app
   	APPD_TIER_NAME: cf-python
     services:
   	- appd

   CODE

6. Push the application.
   

   $ cf push

   CODE

Advanced Configuration

appdbuildpack provides the APPD_CONF_HTTP_URL environment variable to customize agent binary downloads and extend agents advanced configuration respectively. Refer to Multi-Buildpack documentation for details.

GoLang Applications (Multi-Buildpack)

Go applications are pushed using the multi-buildpack approach where the appdbuildpack buildpack is used in tandem with the standard go_buildpack buildpack.

Splunk AppDynamics provides an SDK to instrument such applications. The configuration needed to start the Go agent via the SDK APIs is consumed via environment variables.

1. Edit the following sections of your application’s manifest.yml.

2. Include appdbuildpack in the buildpacks section of application manifest.

3. Set the application environment variable APPD_AGENT to golang.

4. Set the LD_LIBRARY_PATH environemt variable to the folder containing libappdynamics.so.

5. Set the app and tier environment variables based on what’s read by the application using os.Getenv("...").

6. Bind the application to the Splunk AppDynamics service instance in the services section of the application manifest.

   ---
   applications:
   - name: cf-go
     memory: 500M
     buildpacks:
   	- appdbuildpack
   	- go_buildpack
     env:
   	APPD_AGENT: golang
   	APPD_APPLICATION_NAME: my-go-app
   	APPD_TIER_NAME: go-tier
   	LD_LIBRARY_PATH: /app/vendor/appdynamics/lib
     services:
   	- appd

   CODE

7. Copy paste the snippet below to initialize the go agent.
   

   package main
   import (
   	  ...
   	  appd "appdynamics"
   )
   
   	...
   	  cfg := appd.Config{}
   	  cfg.AppName = os.Getenv("APPD_APPLICATION_NAME")
   	  cfg.TierName = os.Getenv("APPD_TIER_NAME")
   	  cfg.NodeName = os.Getenv("APPD_NODE_NAME")
   	  cfg.NodeName = os.Getenv("APPD_NODE_NAME") + ":" + os.Getenv("CF_INSTANCE_INDEX")
   
   
   	  /* Controller Configuration */
   	  cfg.Controller.Host = os.Getenv("APPD_CONTROLLER_HOST")
   	  port, err := strconv.ParseInt(os.Getenv("APPD_CONTROLLER_HOST"), 10, 16)
   	  if err != nil {
   		return err
   	  }
   	  cfg.Controller.Port = uint16(port)
   	  cfg.Controller.Account = os.Getenv("APPD_ACCOUNT_NAME")
   	  cfg.Controller.AccessKey = os.Getenv("APPD_ACCOUNT_ACCESS_KEY")
   	  cfg.InitTimeoutMs = 1000
   
   	  err = appd.InitSDK(&cfg)
   	  if err != nil {
   		  fmt.Println(err)
   	  }
   	  ...

   CODE

8. The necessary configuration to start Splunk AppDynamics, Golang agent is consumed through environment variables APPD_APPLICATION_NAME, APPD_TIER_NAME, APPD_NODE_NAME, APPD_CONTROLLER_HOST, APPD_ACCOUNT_NAME, APPD_ACCOUNT_ACCESS_KEY.

9. Push the application.
   

   $ cf push

   CODE

Advanced Configuration

The appdbuildpack provides the APPD_CONF_HTTP_URL environment variable to customize agent binary downloads, and extend agents advanced configuration respectively. See Multi-Buildpack documentation for details.

Java Applications

Java applications are instrumented using the standard java_buildpack_offline buildpack. The multi-buildpack approach is not used as this is not supported by the Java buildpack.

1. Edit the following sections of your application’s manifest.yml.

2. Set the app and tier environment variables.

3. Optionally set the APPD_CONF_HTTP_URL environment variable to a URL where advanced agent configuration files are hosted. See Extending Agent Configuration documentation for more information. Also see the cf-java-sample and cf-configuration-static applications in the sample app repository for an example of how to override the log4j2.xml file to redirect agent output to STDOUT.

4. Optionally set the APPD_CF_NODE_PREFIX environment variable to prepend a string to the node name assigned by the buildpack.

5. Bind the application to the Splunk AppDynamics service instance in the services section of the application manifest.

   ---
   applications:
   - name: cf-java-app
     memory: 1G
     random-route: true
     path: build/libs/cf-java-sample-1.0.jar
     buildpacks:
   	- java_buildpack_offline
     env:
   	APPDYNAMICS_AGENT_APPLICATION_NAME: my-java-app
   	APPDYNAMICS_AGENT_TIER_NAME: cf-java-app
   	APPD_CONF_HTTP_URL: http://custom-http-server.com
   	APPD_CF_NODE_PREFIX: my-node-prefix
     services:
   	- appd

   CODE

6. Push the application.
   

   $ cf push

   CODE

Advanced Configuration

Because Splunk AppDynamics instrumentation of VMWare Tanzu Java applications does not use appdbuildpack, the configuration is managed by java_buildpack directly.

Refer to the standard java_buildpack documentation.

Node.js Applications (Multi-Buildpack)

Node.js applications are pushed using the multi-buildpack approach where the appdbuildpack buildpack is used in tandem with the standard nodejs_buildpack buildpack.

1. Edit the following sections of your application’s manifest.yml.

2. Include appdbuildpack in the buildpacks section.

3. Set the application environment variable APPD_AGENT to nodejs.

4. Set the app and tier environment variables.

5. Optionally set the environment variable APPDYNAMICS_LOGGER_OUTPUT_TYPE to console to stream agent logs.

6. Optionally set the environment variable APPD_CDN_BASE_URL to point to an internal repo where the Node.js agent package is hosted. This overrides the default Splunk AppDynamics repo that is hosted on the public internet and typically not available in a VMWare Tanzu environment.

7. Bind the application to the Splunk AppDynamics service instance in the services section of the application manifest.

   applications:
   - name: cf-node
     memory: 500M
   env:
     APPD_AGENT: nodejs
     APPDYNAMICS_AGENT_APPLICATION_NAME: my-nodejs-app
     APPDYNAMICS_AGENT_TIER_NAME: cf-nodejs
     APPDYNAMICS_LOGGER_OUTPUT_TYPE: console
     APPD_CDN_BASE_URL: http://internalrepo.com
   services:
   	- appd

   CODE

8. Optionally provide the environment variables for configuring the agent. Below is the list of supported environment variables.
   

   ENV VARIABLE	DESCRIPTION
   APPDYNAMICS_AGENT_APPLICATION_NAME	Application Name to be reported to the Controller
   APPDYNAMICS_AGENT_TIER_NAME	Tier Name for the Application to be reported to the Controller
   APPD_AGENT_DEBUG	Set to true to enable debug level logging for the agent. The default is false.
   APPD_AGENT_LIBAGENT	Controls the proxyless version of the agent. The default is true. Set to false if you want to use the proxy version of the Node.js agent.
   APPD_LOG_CONFIG	The location, level, and other settings related to agent logging. See Node.js Agent Logging.

9. Push the application.
   

   $ cf push

   CODE

Extending Default Configuration

Extending Proxy configuration

If you are using a proxy server, the consumer application should specify the http_proxy or https_proxy as an environment variable for the application running on VMware Tanzu to communicate with the proxy server. Do the following:

1. Set the environment variable for the application and the Controller.

   • For an HTTP proxy: cf set-env http_proxy http://user@password:myproxy....:8080/

   • For an HTTPS proxy: cf set-env https_proxy https://user@password:myproxy....:8080/

2. Restage the application: cf restage <app-name>

3. Log in to the Controller web UI, navigate to the Metric Browser, and verify that the application is being detected and monitored.

Using User-Provided Service

In addition to the procedure above using the service broker tile, operators can also use user provided service with the name appdynamics to instrument applications on Splunk AppDynamics.

1. Use the cf cups command below with the JSON fields specified.

2. The name of the user provided service must be appdynamics and its JSON credentials must have account-access-key, account-name, host-name, port and ssl-enabled fields.

   $ cf cups appdynamics -p '{"account-access-key":"acce$$key", "account-name":"customer1", "host-name":"demo.appdynamics.com", "port":"8090", "ssl-enabled":false}'

   CODE

Deploying the Machine Agent on VMware Tanzu

This topic helps you deploy the Machine Agent on the VMware Tanzu(VMware Tanzu) platform.

Before You Start

• Acquire admin or space developer access on Cloud Foundry to deploy, run, and manage applications. For more information on access rights see the Cloud Foundry Documentation.
• Download the Machine Agent.

Set Up the Machine Agent

1. Set up the GitHub repository.

   git clone https://github.com/Appdynamics/cloudfoundry-apps
   cd cloudfoundry-apps/cf-machine-agent

   CODE

2. Ensure that the Machine Agent is downloaded from the Download Portal into the current directory.
   

   MachineAgent-4.5.11.2163.zip
   cd MachineAgent-4.5.11.2163
   (master)$ ls
   bin			local-scripts		monitorsLibs
   conf		machineagent.jar	readme.txt
   extensions	machineagent.jar.asc	readme.txt.asc
   lib			monitors		scripts

   CODE

   
   

3. Copy the Machine Agent extensions, if any, to the cloudfoundry-apps/cf-machine-agent extensions directory. Note that extensions must be of .zip format, containing extension_folder/{.jar, *.monitor.xml}. For example:
   

   $ cp ~/Downloads/rabbitmqmonitor-2.0.2.zip extensions/

   CODE

4. Edit the JAVA_OPTS section in the manifest.yml to add the Controller information. For example, the manifest.ymlis similar to the following after filling in the details:
   

   ---
   applications:
   - name: appdynamics-machine-agent
   memory: 1G
   health-check-type: process
   no-route: true
   buildpack: java_buildpack_offline
   path: MachineAgent.zip
   env:
     JAVA_OPTS: '-Dappdynamics.agent.accountAccessKey=accessKey -Dappdynamics.agent.accountName=customer1 -Dappdynamics.controller.hostName=mycontroller.com -Dappdynamics.controller.port=<port> -Dappdynamics.agent.applicationName=myAppName -Dappdynamics.agent.uniqueHostId=myUniqueHostId -Dappdynamics.controller.ssl.enabled=false'

   CODE

5. Run package.sh.

   This script packages the extension zip files into the Machine Agent monitor directory, cleans the Machine Agent, and adjusts the permissions of files per requirements from VMware Tanzu.

   $ ./package.sh
   Creating workdir tmp
   Moving MachineAgent to tmp dir
   Unzipping MachineAgent
   Unzipping Extensions from ../extensions to ./monitors Directory
   Cleaning MachineAgent - removing *.asc files
   Adjusting permissions as per requirement from CloudFoundry
   Packaging MachineAgent

   CODE

6. Deploy the Machine Agent using cf push.
   

   $ cf push

   CODE

7. Verify the app is running using cf apps and check the app logs on the container (home/vcap/app) to validate the extension is working.
   

   $ cf apps
   Getting apps in org appdynamics-org / space appdynamics-space as admin...
   OK
   appdynamics-machine-agent   started           1/1         1G       1G

   CODE
   

   The conf/logging/log4j.xml file in MachineAgent.zip can be configured to log to the console (by default it logs to the file system).