A transaction snapshot depicts a set of diagnostic data, taken at a certain point in time, for an individual transaction across all application servers through which the transaction has passed. Transaction snapshots help you identify and troubleshoot the root causes of performance problems.
Code-level visibility is essential for troubleshooting performance problems in production. It gives you details about the exact code path taken by a particular transaction and the time spent in the methods that were executed. A transaction snapshot gives you this code-level visibility for a business transaction, not just at a single node in your environment, but for the transaction as touched by all instrumented tiers involved in processing the business transaction.
How Transaction Snapshots are Generated
A transaction snapshot presents a flow map view of the transaction. In the following flow map for a very slow transaction, you can see that call drill down information has been captured by the originating ECommerce server and by the downstream Inventory server:
Sometimes transaction snapshots are not captured by all the tiers in a distributed transaction. In the next example, transaction snapshots were captured on the originating Apache server but not the downstream ExpandSvc server. Why not?
Here are the rules governing whether transaction snapshots are captured on a tier. The rules are slightly different for originating and downstream tiers. An originating tier is the tier that contains the entry point to the transaction.
Transaction snapshots are captured by the originating tier in a distributed transaction:
- When a diagnostic session is triggered by the originating tier.
The agent starts diagnostic sessions when it detects a pattern of performance problems. In addition you can manually start a diagnostic session from the Business Transaction Dashboard. For details see Capture Details with Diagnostic Sessions.
- When the agent identifies slow, very slow, or stalled response times, or errors on the originating tier.
These snapshots may have partial call graph information, because they start at the time when the transaction slowed or experienced an error.
- Based on the periodic collection schedule.
By default the agent captures one snapshot every 10 minutes. For details see Configure Transaction Snapshots.
Aggressive Slow Snapshot Collection
You can increase the collection of full call graphs by enabling aggressive slow snapshot feature. This may be useful when troubleshooting a specific performance issue, for example. See Slow Snapshot Collection for more information.
Any tier (originating, continuing and terminating tiers) can take a snapshot, thus enabling driil down, when it knows that it is experiencing slow, very slow, or stalled response times or that it has errors.
In addition, any downstream tier captures snapshots if the tier immediately upstream to it tells it to take a snapshot. An upstream tier might direct its downstream tier to take a snapshot under these circumstances:
- The upstream tier is taking a snapshot for a diagnostic session.
- The upstream tier is taking a snapshot based on the periodic collection schedule.
If none of these conditions exists, the downstream tier may not take a snapshot, hence no drill down is available on that tier.
These rules apply even when the downstream tier exists in an application correlated using cross application flow. In order to drill down into the downstream application, a user must be a member of a role with view permissions to the correlated application. See Configure Custom Roles. For information on cross application flow, see "Business Applications" on AppDynamics Concepts.
Bear in mind that in addition to these rules, there is a limit on the number of snapshots per minute per tier, so if this limit is reached, snapshots will not be created for that minute notwithstanding conformity to the rules.
View Transaction Snapshots
You can get a list of transaction snapshots for the selected time range:
- From the Transaction Snapshots tab of the application, tier, node, or business transaction dashboards.
- From the links in the transaction scorecards in the application, tier, node, or business transaction dashboards. The links take you to the snapshots associated with the performance level indicated, Slow, Very Slow, Stall, and Errors.
- From the Troubleshoot-> Slow Response Time or Troubleshoot-> Errors in the left navigation pane
When you double click on any item in the list of transaction snapshots, a dialog appears where you can access the following views for the snapshot:
- Flow Map presents the user experience, execution time, and timestamp of the transaction as a flow map. The flow map also provides details of the overall time that is spent in a particular tier and in database and remote service calls.
- Snapshot Waterfall View presents the call execution times as they occur during the end-to-end transaction time as a chart. The chart shows all events across snapshots associated with the transaction. Asynchronous activity in the waterfall view is distinguished by bar color, as indicated in the legend at the bottom of the page.
- List View shows all snapshots taken for a given transaction as a list. The list is very useful for sorting snapshots according to execution time.
Access the views using the tabs at the top of the dialog:
Transaction Snapshot Call Drill Downs
An individual transaction snapshot contains diagnostic information for individual business transaction instances. It provides information to help diagnose why a particular transaction or series of transactions are not performing as well as expected.
To get the call drill down information:
- In the Transaction Snapshot Flow Map view, click Drill Down on a tier.
- In the Snapshot Waterfall View select a snapshot and double-click or click Drill Down.
- In the Snapshot List View select a snapshot and double-click or click Drill Down.
The contents of a transaction snapshot containing async segments look slightly different if you access the snapshot via the Business Transaction view or via the App/Tier/Server view. In the Business Transaction view, only the originating segments are shown initially, and then you can drill down to the async segments as desired. Because the App/Tier/Server view surfaces all the segments that are relative to that entity, all segments, originating or async, are listed initially.
Diagnostic Data Captured by a Transaction Snapshot in the Call Drill Down Window
The following details are captured in a transaction snapshot:
- Summary: Problem summary, execution time, CPU, timestamps tier, node process ID, thread name, etc.
- Call Graphs: Call graphs in which you can drill down to the method call that is causing the problem. AppDynamics automatically filters non-application classes such as core Java classes, third party libraries, application server and database driver implementation classes. You can configure call graph settings to control which classes should be included or excluded from the call graph. To configure, see Configure Call Graphs.
- Hot Spots: Sorts the calls by execution time, with the most expensive calls at the top. To see the invocation trace of a single call in the lower panel, select the call in the upper panel.
Use the slider to filter which calls to display as hot spots. For example, the following setting filters out all calls faster than 30 ms from the hot spots list.
Using the force-hotspot-if-diag-session and hotspot-collect-cpu node properties you can respectively control whether or not hot spot snapshots are collected for manually started diagnostic sessions and whether CPU time or real time is collected within the hot spot snapshots.
- SQL Calls All SQL queries fired during a request. AppDynamics normalizes the queries and by default does not display raw/bind values. You can configure SQL capture settings to monitor raw SQL data in the queries. Individual calls taking less than 10 ms are not reported.
If you see <n>X in the Query column, the query ran in a batch job n times, with different parameters. The value in the Count column then reports the number of times that the batch job executed.
To configure SQL calls in call graphs, see Configure Call Graphs.
- HTTP Params: HTTP payloads contain basic data such as the URL and session ID, and additional data for Servlet entry points, Struts, JSF, Web Services, etc. You can use HTTP data collectors to specify which query parameter or cookie values should be captured in the transaction snapshot. To configure, see Configure Data Collectors.
- Cookies: The snapshot can use cookie values to help identify the user who initiated the slow or error transaction. To configure, see Configure Data Collectors.
- User Data: User data from any method executed during a transaction, including parameter values and return values, to add context to the transaction. You can use method invocation data collectors to specify the method and parameter index. To configure, see Configure Data Collectors.
In cases where an exit call is made just before a business transaction starts, exit call information can show up in this field, particularly if the transaction is marked as slow or having errors. Please note that sensitive information on the exit call may be shown in this situation.
- Error Details: Exception stack traces and HTTP error codes.
- Hardware/Mem: Graphs for hardware (CPU Memory, Disk IO, Network IO), Memory (Heap, Garbage Collection, Memory Pools), JMX, etc.
- Node Problems: Viewer to analyze node problems. See Troubleshoot Node Problems.
- Additional Data
Transaction snapshots include distributed call graphs and response time distribution details only when a series of bad transactions or a performance policy violation trigger a diagnostic session on a node.
The Controller UI provides several views intended to help you use snapshots to analyze application performance. These are:
- Compare Snapshots shows the performance of calls in two snapshots as a side-by-side comparison.
- Identify the most expensive calls / SQL statements in a group of Snapshots shows the calls that take the most time across the snapshots you have selected.
To compare snapshots, in the Business Transactions All Snapshots list, select two snapshots that you want to compare and click Analyze -> Compare Snapshots.
The Snapshot Comparison window displays a comparison of the selected snapshots. Look at the Time and Change columns to find slow methods.
To analyze the most expensive calls and SQL statements with Snapshots, in the Business Transactions All Snapshots list, select a single or a group of snapshots that you want to analyze. You can select up to 30 snapshots. Then click Analyze > Identify the most expensive calls / SQL statements in a group of snapshots.
Filter Transaction Snapshots
In the All Snapshots subtab of the Transaction Snapshots tab, click Show Filters if the filters are not showing.
You can filter by search criteria before the results are displayed, or you see how many of each criteria are captured, and then refine the list.
The filter attempts to match the criteria that you specify to the values in each snapshot. For example, if you set the search criteria to display error transaction snapshots with a user experience of Error and an execution time greater than 10000ms:
the specified values are compared with the values in each snapshot captured in the selected time range:
Then the snapshots that match the search criteria are displayed in the list:
You can refine the results further by specifying additional criteria. For example, after viewing the previous transaction snapshot list, you might realize that you don't care about snapshots from the /cart/co.GET transaction, so you refine the filter to show only results from the Checkout transaction:
As with other lists in the AppDynamics UI, you can sort the transaction snapshots so that the ones you are the most interested in appear at the top of the list. For example, click and toggle the Exe Time column to display the slowest (those with the longest execution times) at the top. Or click and toggle the Time column to see the most recent transactions at the top.
Normally transaction snapshots are purged after a configurable time - by default, two weeks. To save a snapshot beyond the normal snapshot lifespan – for example, if you want to make sure a snapshot associated with a particular problem is retained for future analysis – archive the snapshot.
Customers with on-premise controllers can modify the default two-week period by configuring the snapshots.retention.period in the Controller Settings section of the Administration console.
- Display the Transaction Snapshot flow map.
- Click the Archive button in the upper right corner.
When you are viewing your snapshot list, a small icon in the far right column indicates that a snapshot has been archived.
To display only archived snapshots in the snapshot list, filter the snapshot list and check Return Only Archived Snapshots. See To filter transaction snapshots using search criteria.