Analytics API Adapters

Before 20.11.0 release, only Legacy adapter was available. As of release 20.11.0, Harmonized adapter is available but remains optional for now.

Analytics API adapters are executed based on t-code /DVD/APPD_CC > KPI detail table. The schemas are only created and populated with data for rows with the Anl (Replicate to Analytics) checkbox set and when SNP CrystalBridge® Monitoring jobs collect KPI values that are linked with detail tables (column Custom event type corresponds to detail table name). 

The link between a detail table and KPI can be found under Assigned to KPI. Click on the MIX values to show a full list of related KPIs.

You can check if a KPI is being monitored in the table of metric mappings under Running profile. If this value is empty, the KPI is not collected, SNP CrystalBridge® Monitoring detail table is not populated with data and no analytics API adapter is executed even when the Anl checkbox is set.

If a KPI is not in the table of metric mappings, t-code /DVD/MON can be used to check if a specific KPI or detail table is processed by an active monitoring job. Right-click on a collector job and select Show monitored KPI(s).

The available Analytics API adapters are listed in the following sections.

Legacy Adapter

This adapter creates custom schemas named according to Schema name value. That means that there will be 1:1 mapping between KPI detail tables marked as relevant for analytics and the event service.

Analytics API limits

The analytics API (Analytics Events API) is limited to 20 custom analytics schemas. This limit is shared among:

• Default detail tables marked as relevant for analytics out of the box.
• Custom detail tables marked as relevant for analytics by customer.
• Schemas created by Business Journeys.
• Custom schemas created by other tools and vendors that use the Analytics Events API.

That means that only up to 20 tables can be used to replicate SAP data to analytics for the whole account. See also Custom Event Ingestion Limits.

This adapter adds the following technical fields to each schema:

• Application - Value customized in t-code /DVD/APPD_CUST.
• Tier - Value customized in t-code /DVD/APPD_CUST.
• Node - Application concatenated with name of application server.
  • If detail table is populated with server (node level) data, this value corresponds to application server where the data originates from.
  • If detail table is populated with system (tier level) data, this value corresponds to application server where monitoring job collected the data.
• SID - SAP system ID.
• TIMESTAMP - Timestamp of the monitoring job run.

Special fields:

• EVENT_TIMESTAMP - When detail table contains this field, it is used to override the default Timestamp (eventTimastamp) field value that is normally populated by the event service.
• USER_EXPERIENCE - When detail table contains this field, User Experience field is added to the custom analytics schema.

Harmonized Adapter

This adapter was introduced to help reduce the total number of custom analytics schemas used by the ABAP Agent. It added a schema grouping model into the KPI detail tables list. When this adapter is used, custom analytics schema name corresponds to Schema group name value. See Mapping Between Legacy and Harmonized Analytics Schemas for a complete list of default schema groups.

When Schema group name is empty, or Schema group is NONE, data will not be send to the event service even when the row is marked as relevant for analytics (Anl checkbox is set). 

Custom row added to the KPI detail tables list can be set as relevant for analytics and use one of schema groups CUSTOM1 - CUSTOM4. This is also possible for overridden default rows where original Schema group value was NONE.

Analytics API limits

It is recommended to distribute custom and overridden tables among these custom groups to avoid custom schema field count limit. See Custom Event Ingestion Limits.

This adapter adds the following technical fields to each schema:

• application - Value customized in t-code /DVD/APPD_CUST.
• tier - Value customized in t-code /DVD/APPD_CUST.
• node - Application concatenated with name of application server.
  • If detail table is populated with server (node level) data, this value corresponds to application server where the data originates from.
  • If detail table is populated with system (tier level) data, this value corresponds to application server where monitoring job collected the data.
• sapTabname - Detail table name. Value corresponds to KPI detail tables row value under Custom event type.
• sapSchema - Schema name previously used by the legacy adapter. Value corresponds to KPI detail tables row value under Schema name.
• SID - SAP system ID

Special fields:

• EVENT_TIMESTAMP - When detail table contains this field, it is used to override the default Timestamp (eventTimastamp) field value that is normally populated by the event service.
• USER_EXPERIENCE - When detail table contains this field, User Experience field is added to the custom analytics schema.

Migrating to Harmonized Adapter

When migrating from legacy to harmonized adapter, the Analytics API limits need to be taken into account. The most critical limit is the 20 custom analytics schemas per account (not SAP per system). If the limit is already reached or not enough custom analytics schemas remain available, switching from legacy to harmonized collector without preparation will lead to loss of analytics data replication.

Once the Analytics API adapter is switched to Harmonized:

1. Legacy custom schemas will retain data recorded before the switch (data retention length depends on number of purchased Transaction analytics licenses) but will not receive new data anymore.
2. Harmonized Adapter will attempt to register new custom analytics schemas. See Mapping Between Legacy and Harmonized Analytics Schemas.
   1. When the harmonized custom schema is successfully created, it will start receiving data instead of the legacy custom schema.
   2. When harmonized custom schema creation fails, the data previously recorded in the legacy custom schema will not be recorded anymore.

To prevent monitoring data loss, consider the following steps to bring down usage of custom analytics schemas. The harmonized adapter requires at least 6 free custom analytics schema resources (+1 when BiQ collector is used and +1 when custom tables are replicated to analytics).

1. Temporarily disable Analytics events API via t-code /DVD/APPD_CUST on non-productive SAP systems. 
2. Check for custom schemas that are not used by productive SAP systems and delete them.
3. If not enough custom analytics schema resources are available, please consider temporarily disabling replication of unused or lower importance legacy schemas in productive SAP systems.
4. Delete also the deactivated productive legacy schemas (right away, or after data retention policy removes all data from the schemas).
5. Switch a non-prod SAP system to harmonized adapter and activate Analytics events API via t-code /DVD/APPD_CUST to make sure that all harmonized custom analytics schemas will be registered correctly.
6. Once all harmonized schemas are registered, switch all productive systems to harmonized adapter and validate that all the schemas get populated with data.
7. Once productive SAP systems are migrated, you can migrate also non-productive SAP systems and turn Analytics events API back on.

Migrating Analytics Dashboards and Metrics

When migrating from legacy to harmonized adapter, the legacy schemas will stop receiving data and some may be deleted during the migration to free up limited resources.

Please check what custom schemas are used in dashboards or analytics query metrics. Any dashboards that use the legacy custom analytics schemas will need to be adjusted manually. Any metrics created from the legacy custom analytics schemas will need to be recreated. This is especially important if such metrics are used for alerting as the legacy metric will eventually stay on 0 after the switch.

Data replicated to harmonized schemas will contain legacy schema name in sapSchema field to enable easy migration of existing dashboards. To switch from legacy to harmonized schemas in dashboards and metrics:

1. Check Mapping Between Legacy and Harmonized Analytics Schemas to determine old and new schema name
2. Edit the query behind the dashboard widget or metric and carry out the following changes:
   1. Replace the old (legacy) schema name with the new (harmonized) schema name in the FROM part of the query string.
   2. Add an extra WHERE condition AND sapSchema = <old schema name>.

Example:

SELECT * FROM idocs_details WHERE SID = "ED2"

SELECT * FROM sap_idoc_data WHERE SID = "ED2" AND sapSchema = "idocs_details"

Deleting Custom Analytics Schemas

From the 21.11.0 release onwards, it is possible to delete custom analytics schemas that were generated by ABAP Agent. 

Prerequisites

1. Analytics Events API Settings must be maintained.
2. OAuth Settings must be maintained.

To delete unused custom analytics schemas, navigate to t-code /DVD/APPD_STATUS, expand the Event service node. After you expand the Custom Analytics Schemas node, the full list of custom analytics schemas that were generated by the ABAP Agent will be loaded. A delete button is available next to an unused schema name. 

Click the delete button and confirm the pop-up prompt to remove the schema from the event service.

You can click on Used status text to see the list of SAP systems that populate the schema with data.

Deletion of used schemas

You can delete schemas that still contain data by activating the Debug Mode. This change is irreversible, and once the pop-up prompt is confirmed, the schema and all data that it contains will be deleted from the event service.

Troubleshooting

• Schema is empty or does not exist

  • Check if SNP CrystalBridge® Monitoring collectors are running successfully

  • Check analytics adapter settings in /DVD/APPD_CUST if you are using Harmonized or Legacy version

  • Make sure you are looking for data in the correct schema analytics schema depending on your version. Since the 20.11 release, multiple schemas were merged into schema groups in order to reduce the number of schemas on the controller. This mapping can be found here.

  • If you are looking up the correct schema and still don’t see any data, it may be because you moved from the Legacy version to Harmonized and new schema groups could not be registered due to the analytics schema count limit:

    • If you are using ABAP Agent version 21.11 and newer - you can delete unused schemas directly from transaction /DVD/APPD_STATUS following this guide

    • If you are on a version older than 21.11 - contact support for schema deletion assistance