To begin monitoring IIB, you must install the IIB agent.

Install the Agent

  1. Download the agent from https://download.appdynamics.com/download/.
  2. Extract the IIB agent zip file into your chosen installation directory.

For cURL installation, see Download AppDynamics Software.

Configure the IIB Agent

The page guides you on how to integrate Appdynamics IIB Agent on Linux and AIX. The installation steps for IIB agent for IIB v9 and v10 are the same for Linux and AIX. The installation steps for IIB, ACE 11, and ACE 12 are provided in a separate tab.

ACE is currently supported on Linux and AIX v7.2.

  1. Specify the settings for the agent-controller communication. In the controller-info.xml file, set these properties:

    • account-name: Account name of the controller login.
    • account-access-key: Access key. This key is used for verification with the controller.
    • application-name: Name of the application that the IIB server belongs to.
    • controller-host: Hostname of the controller.
    • controller-port: Port number of the controller.
    • controller-proxy-host: Host name or IP address of any proxy required for the agent to connect to the controller.

    • controller-proxy-port: Port number of any proxy required for the agent to connect to the controller.

    • controller-proxy-username: Username for authenticating with the proxy.

    • controller-proxy-password: Password for authenticating with the proxy user.

    • controller-proxy-passwordfile: Full path name of a file containing the password for authenticating with the proxy user.

    • controller-ssl-enabled: SSL login. Set 1 to enable, 0 to disable.
    • controller-cert-file: Full path to the PEM format X509 certificate for SSL.
      See, Enable SSL for the C/C++ SDK for more information on obtaining the file.
    • log-dir: Path to the directory containing the IIB agent log files. The default path is /tmp/appd. Logs that are written before this configuration are logged in this path. 
    • log-level: Level of the logs. Set this property to trace|debug|info|warning|errorError is the highest priority and trace is the lowest priority.

    • tier-name: Name of the tier representing the broker.
    • user-exit: Exit name of the user. This must be in the alphanumeric format, as provided to the mqsichangebroker command.
    • <log-settings> </log-settings>: Log settings of the agent. For more information, see Configure Agent Log File Size.
    • disable-correlation: Enable or disable correlation of specific backends. Set 1 to enable, 0 to disable.
      disable-http-correlation: Set 1 to enable, 0 to disable.
      disable-jms-correlation: Set 1 to enable, 0 to disable.
      disable-mq-correlation:  Set 1 to enable, 0 to disable.
    • flow-level-visibility-enabled:  Flow level visibility option. Set 1 to enable, 0 to disable. The default value is 0. See IIB Agent Flow Level Visibility.
    • <node-reuse>true</node-reuse>: Set this property to reuse the node names of historical VMs for new VMs. It prevents the rapid increase of differently named nodes. See Enable the Node Name Reuse.
    • <node-reuse-prefix>prefixName</node-reuse-prefix>: Use this property when node-reuse is set to true. If you do not set this property, the IIB agent generates node names as per the internal node name standards of IBM. See Enable the Node Name Reuse.
  2. Run the following command to stop the broker. To configure user exits, your broker must be stopped.

    mqsistop <broker_name> 
    CODE
  3. Install the IIB agent user exit. Ensure that the IIB agent user exit is enabled by default, for all the flows. Perform the steps based on the version applicable to you.

    Run the following command:

    mqsichangebroker <broker_name> -x <install-directory> -e <user_exit_name>
    CODE

    Perform the following in the node.conf.yaml file. You can find the file in the path: <path-to-installation-directory>/<broker-name>/node.conf.yaml.

    UserExits:

    • activeUserExitList: '<User Exit Name>'   #Specify the name of user exit
    • userExitPath:'<path>' #specify the path.

    The agent starts and stops with the broker, if it is installed on the broker.

  4. Run the following command to start the broker.

    mqsistart <broker_name>
    CODE

For IIB 10, you can monitor a subset of your message flows. You can exclude flows that are of less importance from being monitored by running the following command:

mqsichangeflowuserexits <broker_name> -e <integrationServerName> -f <MessageFlow> -k <application_name> -i <user_exit_name>
CODE

Partial Instrumentation of Brokers

If you have a high volume of activity flow in your IIB instance, you may want to select only the critical flows to instrument. Perform the following to instrument flows on an opt-in basis:

  1. Install the agent with the following command to instrument flows on an opt-in basis.

    mqsichangebroker <broker_name> -x <install-directory>
    CODE
  2. Add the agent to individual flows with the following command:

    mqsichangeflowuserexits <broker_name> -a <user_exit_name> -e <integrationServerName> -f <MessageFlow> -k <application_name>
    BASH

An application node registers for each broker process that has one or more instrumented flows deployed to it. Business transactions are only created (or continued) for flows where the agent's user exit is active.

If you have a high volume of activity flow in your ACE instance, you may want to select only the critical integration servers to instrument.
To instrument integration servers on an opt-in basis, edit the following in the server.conf.yaml file. You can find the file in the path "<path-to-installation-directory>/<broker-name>/servers/<server-name>/server.conf.yaml.

  1. UserExits:
    • activeUserExitList: '<UserExitName>' #specify the name of an installed user exit to activate.

    • userExitPath: '<path>' #specify the path.

  2. Set the following parameters in the ‘UserExits’ configuration of the node.conf.yaml as blank.

    UserExits:

      • activeUserExitList: '' #set it as an empty string.
      • userExitPath: '' #set it as an empty string.
  3. Restart the broker.

AppDynamics Agent is enabled only for those integration servers that have been instrumented and not for all the servers. Business transactions are only created (or continued) for integration servers where the agent's user exit is active.

Configure Agent Log File Size

AppDynamics IIB Agent creates a separate log file for each execution group. Therefore, each log continuously grows, even if the execution group is not instrumented.

Hence, a need to stop pushing the updates to the logs for non-instrumented execution groups arises. AppDynamics IIB Agent now allows you to configure the log file size according to your requirement.

The following setting in the configuration file, controller-info.xml allows you to configure the log file size of the agent:

<log-settings>
<log-setting broker="" execution-group="">
<max-file-size></max-file-size>
</log-setting>
</log-settings>
CODE

The settings are defined as follows:

SettingDescription
<log-settings></log-settings>Header tag
<log-setting broker="" execution-group="">Broker name and the execution group name, which must be strings
<max-file-size> </max-file-size>The maximum size of the log file. The format is <number><unit>. For example, 5 MB. Valid units are "MB" or "KB". The accepted range in KB is [200KB, 999KB] and in MB it is [1MB, 999MB].  Every execution group has its own log file.

However, you need to note the following guidelines for the configuration:

  • If a broker is not named, the setting applies to all the brokers.
  • If an execution group is not named, the setting applies to all the execution groups.
  • An execution group name must be accompanied by a broker name.

If you include the log setting, log-rotation is set to 1, else the default value of 5 is considered as the log rotation.

For example, in the following configuration, there are three instances of log setting:

<controller-info>
<controller-host></controller-host>
<controller-port></controller-port>
<controller-ssl-enabled></controller-ssl-enabled>
<account-name></account-name>
<account-access-key></account-access-key>
<application-name></application-name>
<tier-name></tier-name>
<user-exit></user-exit>
<log-level></log-level>
<enable-extended-logging></enable-extended-logging>
<controller-cert-file></controller-cert-file>
<controller-cert-dir></controller-cert-dir>
<log-settings>
<log-setting broker="" execution-group="">
<max-file-size>200KB</max-file-size>
</log-setting>
<log-setting broker="Broker1" execution-group="EG1">
<max-file-size>2MB</max-file-size>
</log-setting>
<log-setting broker="Broker1" execution-group="">
<max-file-size>3MB</max-file-size>
</log-setting>
</log-settings>
</controller-info>
CODE

The order of preference is mentioned in the following table. The preference order is defined as 1 → 2 → 3 → 4. Hence, if an order is invalid or does not match, then the next valid order is considered.

Order NumberPreferencesOrder Template
1Execution-Group<log-setting broker="Broker1" execution-group="EG1">
2Broker<log-setting broker="Broker1" execution-group="">
3Across multiple-broker<log-setting broker="" execution-group="">
4DefaultAgent default is applied which is 10MB

The following are the valid values, for different execution groups when used with the above example configurations.

Broker Name

Execution Group Name

Valid Values (max-file-size)

Broker1EG12MB
Broker1All other EGs3MB
All other BrokersAll other EGs200KB

Enable the Node Name Reuse

The node name reuse function enables users to reuse the node names of historical VMs for new VMs. It prevents the rapid increase of differently named nodes, especially when the nodes are identical processes that run over different times. This helps you monitor environments with short-life VMs. 

To enable this function, specify the following in the IIB configuration (controller-info.xml) file.

<node-reuse>true</node-reuse>
<node-reuse-prefix>prefixName</node-reuse-prefix>
CODE

The node-reuse-prefix property allows the controller to generate node names dynamically using the prefix <prefixName>. The nodes running in parallel are differentiated by suffixes (-1, -2, -3, and so on), depending on the number of nodes running. It generates a node name with App, Tier, and Sequence number when enabled. It also pools the node names. For example, the sequence numbers are reused when the nodes are purged (based on the node lifetime).

Once a node shuts down, it becomes a historical node, and its name can be used as a new node. However, if a node shuts down unexpectedly or forcefully, the agent does not inform the controller about the shutdown. In this case, the node name is not reused. To avoid forceful shutdowns,  you can use terminationGracePeriod. IIB agent recommends terminationGracePeriod of 60 seconds.

For more information on terminationGracePeriod, see Kubernetes best practices: terminating with grace.


If you do not specify this function, the IIB agent generates internal node names based on IBM standards.

Troubleshooting

If you experience any functional issues with the agent, review the following information and supply it to the AppDynamics Support.

  • The agent log. The IIB agent writes status information into the configured log directory, or /tmp/appd if the configuration has not been read at the time the message is logged. The status information can help diagnose issues with the agent.
  • The system log. The IIB broker writes status messages into the system log. The messages include information about the loading and operation of user exits, such as the AppDynamics IIB agent.
  • Abend files in the IIB errors directory. The default location is /var/mqsi/common/errors.
  • Core files, if they exist.

If the user-exit name provided in the controller-info.xml file is not alphanumeric, the broker cannot load the agent and fails to start.

If you have an issue with the agent or if you want to disable it, perform the following:

  1. Stop the broker mqsistop <broker-name>.
  2. Perform the following based on the version applicable to you.

    Remove the AppDynamics user exit mqsichangebroker <broker-name> -x "" -e ""

    Set the following parameters in the node.conf.yaml file as an empty string.

    UserExits:
    activeUserExitList: ''
    userExitPath: ''

  3. Start the broker mqsistart <broker-name>.

Disable the IIB Agent

If you have enabled the agent on a flow-by-flow basis, perform the following for each flow the agent is enabled on.

  1. Stop the broker mqsistop <broker-name>.
  2. Disable the user exit for each flow mqsichangebroker <broker-name> -a "" -e <integrationServerName> -f <messageFlow> -k <applicationName>.
  3. Start the broker mqsistart <broker-name>.

If you have enabled the agent on integration server level, perform the following for each servers.

  1. Stop the broker mqsistop <broker-name>.
  2. Set the following parameters in the server.conf.yaml file as an empty string. You can find the file at the location: "<path-to-installation-directory>/<broker-name>/servers/<server-name>/server.conf.yaml.
    UserExits:
    activeUserExitList: ''
    userExitPath: ''
  3. Start the broker mqsistart <broker-name>.

Turn off MQRFH2 Injection Added by AppDynamics Monitor Agent Tool

For IIB Agents that use the MQ mechanism, the MQRFH2 message header is used to provide correlation to downstream tiers. Any applications consuming MQ messages from an upstream IIB agent must support the MQRFH2 header standards to provide end-to-end correlation.

In a situation where the downstream agent reports the error MQRC_HEADER_ERROR (MQRC 2142) while parsing the message from an IIB agent via MQ, you can perform one of the following:

  • Update the downstream consumer to accept the MQRFH2 header structure by upgrading the downstream software to a version that supports MQRFH2.
  • Disable MQ correlation in IIB Agent by setting <disable-mq-correlation>1</disable-mq-correlation> in the controller-info.xml file and restart the agent.
  • Change the MQ PROPCTL property to NONE on the destination queue.  See https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_7.5.0/com.ibm.mq.mig.doc/q008230_.htm

Troubleshooting for IBM MQ .NET Clients

AppDynamics Monitor Agent tool adds an additional RFH header to the MQ message before adding the message to the queue. Because of this additional RFH header, IBM MQ .NET Clients may report an MQRC_HEADER_ERROR (MQRC 2142) while parsing the message. Following is the sample RFH header that gets added to MQMessage.

RFH ….x…”…3…MQSTR ……..P…<usr><singularityheader
dt=”string” >notxdetect
=True</singularityheader></usr>

The RFH2 structure begins with <usr><singularityheader dt=”string” >. This RFH header is added by the AppDynamics Monitor Agent tool and not by MQ.

Perform these steps to disable the additional header on the tool:
1. Go to Configuration > Instrumentation > Backend Detection.
2. Click on your application name, go to the .NET tab and click Queues.
3. Click Edit Automatic Discovery and in the popup, clear the Correlation Enabled checkbox.