In AppDynamics, databases and remote services (and in fact, any detected out of process components that are involved in Business Transaction processing) are collectively known as backends.
AppDynamics discovers backends from exit point instrumentation placed in the application code. An exit point is the precise location in the code where an outbound call is made from an instrumented node. AppDynamics automatically discovers a wide range of backends.
Default Automatic Backend Discovery
A discoverable backend is identified by its type and associated properties. The precise set of properties used to identify any given backend is dependent on its type. The type itself is defined in the built-in backend discovery rules for backend types supported out of the box. For exit calls to backends that are not instrumented with APM agents, AppDynamics uses the backend properties to identify and name the backend. Where a backend call is actually processed by a downstream tier also instrumented with AppDynamics, ensure that any given backend that is identified will always result in calls that will be processed by just one downstream tier. This means, for example, that if tier A makes an HTTP call to an http router on localhost:4040 which will forward the call to tier B or tier C depending on the URL, custom backend naming rules will be required to include enough segments of the request URL - as well as the host and port - such that requests destined for tier B will be associated with a backend with a different set of backend properties than those associated with tier C.
The Automatic Backend Discovery list in the Configuration > Instrumentation > Backend Detection tab shows the configurable backend discovery rules.
The automatic discovery rules vary according to the type of backend being identified, but generally include settings for enabling automatic discovery of the backend type, enabling correlation, and properties used to identify and name the backend. AppDynamics uses transaction correlation to track request processing across distributed tiers. Certain types of backends are automatically discovered as high volume exit points (sometimes called turbo exit points). These backend types include cache servers, EhCache, Danga Memcache, Memcached, and Oracle Coherence. For more about high volume exit points, see Exit Point Detection Rules.
When an exit point is identified for the first time, the exit call results in a backend discovery event.
To modify backend detection, you need the Configure Backend Detection permission.
Modify Automatic Discovery
If you know that the application flow map should be showing a specific backend and you are not seeing it, you can modify the configuration to detect backends. You can:
- Edit the default backend discovery rules.
- Create new custom backend discovery rules.
- Create new custom exit points to provide visibility for backend types that are not automatically discovered.
- Disable automatic discovery of backends that are not of interest.
- Disable correlation for the backend.
In many cases, you can achieve the level of customization that you need by editing the default rules rather than creating new rules.
To edit a rule, in the Backend Detection tab, select the application and click the Edit Automatic Discovery button.
For certain backend types—particularly those named by URL—the name of the discovered backend instances include a port number. If the app agent is unable to discover the port number for some reason, the port number is displayed in the UI as "-1". This can happen, for example, when a port is not a public port or the API for discovering the port is not available on the application environment (often the case for an RMI backend type). Despite having an unknown port number, the backends are nonetheless uniquely identified.
Configurable Properties and Naming
For many of the automatically discovered backend types, AppDynamics uses a set of configurable properties to identify and name the backend. For example, the properties for a JMS messaging queue are something like this:
- DestinationType (for example, QUEUE)
- Destination (for example, OrderQueue)
- Vendor (for example, Active MQ)
The following informational popup shows how the values are used to identify a discovered backend. In the following example, callout 1 shows the value that corresponds to the destination type, callout 2 to the vendor name, and callout 3 to the destination queue name:
Aggregate or Split Backends
You can change the default configuration for the backend types shown in the UI to aggregate or split the backends. For example, you could disable the Vendor property for JMS backend detection to aggregate (join) all JMS backends with the same destination and destination type. The metrics for all JMS backends with the same destination and destination type would be aggregated as one backend.
For properties that you want to use, you can also configure how AppDynamics uses the property. For example, if the URL property is used to identify a backend and your URLs include a file name, the default configuration results in a separate backend for every file. To reduce the number of backends identified using the URL property, configure which parts of the URL to use. For example, run a regular expression on the URL to discard the file name or any other parts of the URL that you do not want used for identification.
Add Custom Backend Discovery Rules
AppDynamics provides the additional flexibility to create new custom discovery rules for the automatically discovered backend types.
To create a custom discovery rule for an automatically discovered backend type, select the backend type in the Automatic Backend Discovery list. Click Add (the + icon) to create a new rule. Specify the following:
- Name for the custom rule.
- Enabled to enable or disable the rule.
- Correlation Enabled to specify whether or not to send business transaction correlation data to the backend.
- Priority for this custom rule compared to other custom rules for this backend type. The higher the number, the higher the priority. A value of 0 (zero) indicates that the default rule should be used.
- Match Conditions used to identify which backends are subject to the custom naming rules. AppDynamics applies the default rules to backends that do not meet all the defined match conditions.
- Backend Naming Configuration used to name the backends matching the match conditions. The naming configuration must include the property used by the match condition.
See specific exit points for examples.
All Other Traffic Backends
AppDynamics limits the number of backend systems that can be registered and tracked. Once the limit is reached (databases and remote services combined), additional backend are put in the "All other traffic for <type>" category.
The "type" is the backend type such as HTTP or Web Service. HTTP calls and JMS queues are the most common types that may reach the limit with the default configuration rules.
On a flow map, the category appears as follows:
Metrics for the backend calls grouped in the "All other traffic" category are aggregated. If the group includes backend instances that you want to be individually tracked, you can manage backend detection, as described in Manage Backend Discovery.
Backend Registration Limits
The limits on the number of backends that can be registered:
- The Controller enforces a limit of 1000 backend registrations in the business application for each type of backend (that is, 1000 HTTP backends, 1000 JMS backends, and so on). This setting is controlled by the
backend.registration.limitController Setting. This limit applies whether the backends are resolved to a tier or unresolved.
- Agent apply a limit of 300 unresolved backends per application.
When the backend count exceeds the limit, the "All Other Traffic" category is automatically created. The logs indicate that a limit has been reached by a log entry such as the following:
Manage Backend Discovery
After reaching the backend registration limit, you should make sure that the systems being tracked as first-class registered backends are the ones that are important to you relative to those in the "All Other Traffic" category.
Also make sure that application environmental conditions aren't causing an excess number of backends to be registered. These conditions may include:
- JMS queues that use the session ID in the destination. This causes each session to be identified as a separate backend.
- Calls to message queues hosted on the same server. In this case, you might not be interested in seeing each queue separately, and instead, want to aggregate everything for the same host and port to the same backend.
- Dynamic generation of IP addresses, such as when you host a service on Amazon Elastic Compute Cloud (Amazon EC2).
- Different JDBC drivers connecting to the same database may cause many backends. This can happen when there are slight variations in the driver metadata that AppDynamics uses to identify the database. For example, if two different JDBC drivers are used to access the same database, something as simple as a different spelling or format of the database name (ORACLE, oracle, or ORACL) can generate multiple database names when they are the actually the same physical database.
To manage backend registrations, you can change the configuration of the backend detection rules. For example, if a property causes excessive unique identification, consider removing the property or modifying its use in the detection and naming rule. If you created custom exit points, consider refining or removing them.
Delete Unnecessary Backends
Some backends may no longer have traffic. You can delete unused backends from the Remote Services List, any Remote Service dashboard, the Databases List, and any Database dashboard. If the backend has new traffic in the future, the app agent rediscovers it and reregisters it with the Controller.
You can also configure the Controller to automatically remove stale backends. See Stale Remote Service Removal.
Organize Backends in Flow Maps
Once you have configured detection for all the backends you require, you may need to organize the way they appear in the UI. See Group Remote Services on Flow Maps.