Set Performance Boundaries Using Thresholds

This topic describes how AppDynamics thresholds help you to maintain the service level agreements (SLAs) and ensure optimum performance levels for your system.

Thresholds provide a flexible way to associate the right business context with a slow request to isolate the root cause.  The following areas of the platform use thresholds:

• Transaction Scorecard
• Diagnostic Sessions

By default, every business transaction is evaluated against three different thresholds for classification as:

• Slow
• Very Slow
• Stall

When a business transaction is originally detected, it inherits all these thresholds from the application. Threshold values can be overridden for an individual business transaction. You can configure these thresholds. Every transaction is classified as either a normal, slow, very slow, stall or error transaction. Which means that:

• Calls = Normal + Slow + Very Slow + Stall + Error

Once a transaction is an error, it is not also categorized based on response time. This is why the Transaction Scorecard also shows error transactions as seen in the following screen capture:

Applying Thresholds

You can apply thresholds at following levels:

•  Application
• Individual business transactions
• Background tasks (long-running transactions)
• Diagnostic sessions

Static and Dynamic Thresholds

A static threshold is a static value, which when exceeded, violates the threshold.

A dynamic threshold is based on a changing value. You can specify dynamic thresholds using either percentage deviation or standard deviation measures.

Percentage Deviation

Percentage deviation defines a threshold based on the moving average of the requests over a certain interval. Default time interval is 2 hours. If the average response time for the last two hours is X milliseconds, and if the request takes the percentage deviation over X ms, the transaction violates threshold.

For example, a transaction occurs at 10:00:01 AM. The average of the requests between 8:00:00 and 10:00:00 is 100 ms and the slow threshold is defined as 20 % over threshold. If the transaction takes 120 ms, it will be considered a slow transaction.

Standard Deviation

Standard deviation defines a threshold based on the moving average over a certain interval. Default time interval is 2 hours. This means if the average response time for the last two hours is X milliseconds, and if the request takes the standard deviation over X ms, it violates threshold.

For example, a transaction begins at 10:00:01 AM and the slow threshold is defined as 3 times the standard deviation (variation from the mean). If the transaction takes more than 3 standard deviations, it will be considered a slow transaction.

How is a transaction marked slow, very slow, stall, or error?

As soon as a transaction starts, the particular thread representing that transaction is monitored for a stall. Every 5 seconds, the inflight transactions are evaluated to determine if they have met or exceeded the stall threshold. Obviously if the transaction finishes before the value of the stall threshold for a BT, it is not marked as a stall.

If a transaction is not a stall and finishes before the stall threshold, it is compared against the very slow threshold first and then the slow threshold and marked accordingly.  If a transaction hits the stall threshold (takes more than 300 deviations above the average for the last 2 hours or the set stall threshold), a stall transaction event is sent out because the transaction might take a very long time to eventually finish or time out.

Types of thresholds

There are two types of thresholds

• static
• moving average based

On every node of the starting tier of the business transaction, a moving average as well as the standard deviation is maintained for a specified time window (by default the last 2 hours). A specific transaction response time can be compared based on its standard deviation or  a percentage slower of the moving average.

The graphic shows a two hour moving average window, from 10 : 00 AM to 1: 00 PM. All transactions that come in during 1:00 PM and 1:01 PM are measured against the response time and standard deviation of this window. For the next minute, between 1:01 and 1:02 PM, the window moves to 10:01 AM to 1:01 PM.

• Data for minutes where there is no load is not counted.
• This is maintained locally on each node for the starting tier for a business transaction. The controller does not hold any of this data.

You can configure the threshold evaluation against a percentage of the average, a static value, or a multiple of the standard deviation.  

To understand how a standard deviation threshold works, consider a moving average that is 1500 ms with a standard deviation of 100 ms. If you set the threshold to 3, as in the following example, it means that the threshold will be three times the standard deviation. In other words, a transaction that takes more than 1500 + (3*100) or 1800 ms will violate the threshold.

Alternatively, you can set the threshold to a static value or use a percentage of the average.