On this page:

Related pages:


BiQ Collector for Monitoring Integration is designed to monitor SAP business processes. It is an SNP CrystalBridge® Monitoring collector built specifically for capturing information on the SAP Business process documents. The collector runs in the SAP System at regular intervals, and collects the configured information on the configured documents that are created or updated since its previous run. 

  • Sales orders, delivery documents, invoices, and other documents that are being created and updated on the SAP system(s) are monitored as business process steps.

  • Related documents of individual business process steps (e.g. Sales order and it’s Delivery document(s)) are connected together using Global unique ID (GUID).

  • Any information related to any of the documents can be captured and is configurable.

  • The individual business process steps or document types are captured using either:

    • Pre-built implementation that can cover simple cases like “Here is a table or view and here are the fields to capture“.

    • Own or Custom ABAP code to cover more complex cases.

The collected information is propagated to AppDynamics Analytics schema sap_biq_documents. Business Journeys and other dashboard widgets are populated from this schema.

As of version 21.2.0, the cross-system setup allows tracking documents in the multiple SAP ABAP systems, connecting them with the same GUID, and so they are visible in the Controller as a single document flow.

Prerequisites

  • SAP NW ABAP system 701 SP 15 and newer. See SAP Supported Environments for details.
  • The current version of ABAP Agent fully working and integrated with AppDynamics Controller and Transaction Analytics. See Connect to AppDynamics Agents for details.
  • The current version of SNP CrystalBridge® Monitoring is fully working and integrated with AppDynamics Controller and Analytics. See Monitoring Integration for details.

Initial BIQ Installation 

Perform the following steps to install BiQ collector:

  1. Ensure that the target SAP system meets the requirements mentioned in the previous section.

  2. Import the transport request with the BiQ Collector software.
    Make sure to import the transport(s) into the logon client.

    As of release 23.2.0, BiQ Collector is included in ABAP Agent CORE and ABAP Agent 740 installation transport requests, and it no longer needs to be installed separately.

  3. Navigate to transaction /DVD/APPD_BIQ_SETUP.

  4. Click Install BiQ Collector button to register the BiQ Collector into the existing SNP CrystalBridge® Monitoring installation.

  • The installer does not register the BiQ Collector into SNP CrystalBridge® Monitoring if the BiQ Collector (including the Beta version of the collector) is running at the moment.
  • If the Beta Collector is registered (profile ZAPD_BIQ), this installer un-registers the collector and registers the APD_BIQ instead.
  • Please note that at this point, the BiQ Collector is registered and set to be replicated to Analytics, but not yet running.

Local Setup

Local settings allow the operator to set up handling of the GUIDs for captured documents.

  1. Navigate to transaction /DVD/APPD_BIQ_SETUP.

  2. Click the Local setup button.

  3. In the pop-up that appears, local setup can be updated as follows:

    1. GUID retention (time in days) - Allows the BiQ installation to get rid of the old GUIDs for documents that are no longer important and not expected to continue with other business steps. This mechanism prevents the BiQ collector to cumulate the metadata infinitely. If not set or set to 0, the default retention period of 60 days is used.

    2. Master System RFC - For cross-system monitoring setup, the local installation uses this RFC destination to ask for a new, consolidated GUID.

Setup Business Process Monitoring 

To monitor SAP business process, perform the following setup steps: 

New harmonized UI

As of version 22.5.0, a new universal UI for business process setup is available. The new UI allows defining table-based as and IDoc-based business process steps. See New Business Process Setup UI.

  1. Navigate to transaction /DVD/APPD_BIQ_SETUP.
  2. Click Business process maintenance to set up Business processes and Business process steps.
    1. You can set Business processes as shown below:

      • Business process - Technical name of the business process 

      • Short text - Description of the business process.

      • Active flag - Only active business process will be processed by the BIQ collector. This checkbox indicates if this business process should be processed by the BIQ collector.

    2. You can set Business process steps within Business processes as follows: 

      • Business process - Technical name of the business process where this business process step is being added.

      • BP Step - Technical name of the business process step (needs to be unique within a business process)

      • Sequence - Sequence of this business process step in the business process. 

      • BP Step ABAP Class - Class implementing the monitoring of the particular step. This is where custom implementation can be registered. See the following sections for more details.

      • Short text - Description of the business process step.

      • Active flag - Only active business process steps will be processed by the BIQ collector. This checkbox indicates if this business process step should be processed by the BIQ collector.

Checking and Testing Configuration

You can check the current configuration by clicking the Check configuration button in the transaction /DVD/APPD_BIQ_SETUP.

The existing configuration can be also tested, using the Test configuration button in the transaction /DVD/APPD_BIQ_SETUP, which instantly collects and shows the data in the time-frame of the last hour without persisting anything (so nothing is sent to the Analytics nor stored anywhere):

Configuration Logistics

The exiting BP configuration can be transferred to other systems in 2 ways:

  • SAP standard TMS - config table contents are written into a workbench request in the development/sandbox environment that can be then pushed to higher systems via SAP Transport management system.

  • Import/export local JSON file - this is handy in case of some ad-hoc configuration changes e.g. in the quality environment, that are closed for changes.

SAP Table Based Business Process Steps (“Simple Cases”)

The default BiQ ABAP class /DVD/APPD_BIQ_TABLE_STEP is the default implementation that can be used to monitor basic, table-based business process steps.

Identify a Simple Case BP Step

  • The document is based on an Open SQL select statement (i.e. the document data are stored in SAP Tables).

  • The monitoring information does not require any complex conversion logic beyond the capabilities of the value expressions (see below).

  • The entries can be monitored in one of the two approaches:

    • The table/view has a Date field and a Time field that can be used to select documents relevant for the given time-frame. Optionally, another pair of "Update" Date/Time fields can be registered.

    • The table is read in the "timeless" mode as a whole at each collector execution (custom WHERE condition can be specified).

  • The document can be connected to up to one "previous step document". E.q. Sales order in the table VBAK and the Delivery note in table (line item) LIPS are connected via LIPS-VGBEL = VBAK-VBELN. 

If the above conditions are met for the particular business process step, it can be monitored using the /DVD/APPD_BIQ_TABLE_STEP implementation, without additional ABAP coding.

As of release 20.11.0, the simplified UI is introduced to enable setup of the complete process in a single screen that allows the user to edit the complete process:

  • BP and its steps
  • Table based Document definitions
  • Custom fields

Setting Up Table Based BP Step

  1. Navigate to transaction /DVD/APPD_BIQ_SETUP.

  2. Click Business process maintenance button.

  3. Click New BP button, fill the required info and confirm to create e new Business process: 

    1. Business process name - Technical name.

    2. Short text - Description of the Business process.

    3. Active flag - Only active Business processes are monitored.

  4. Click Display setup button

  5. Add or Configure BP steps via the right-click menu.

    1. Business process

    2. Business process step name - unique step name within a BP.

    3. Sequence - sequence of the step.

    4. BP Step ABAP Class - class implementing the monitoring of the particular step. This is where custom implementation can be registered. To choose the default implementation, pick the ABAP class /DVD/APPD_BIQ_TABLE_STEP.
    5. Short text- brief description of this step.
    6. Active flag - only active steps are monitored
  6. Press the Save button or hit Ctrl+S.
  7. Configure the Document definitions for the BP steps:

    1. Business process step name

    2. Document type - technical name that identifies the type of the document. This will be propagated into the resulting Analytics schema.

    3. Document table (or View) - table or view name where the document records are stored. Alternatively, the freeform SQL FROM statement can be created to avoid the necessity of creating views, when documents contains data from more than a single table. The GUI to do this can be invoked by double-clicking the particular Document table column cell.

      Cluster table support

      As of release 22.2.0, BIQ collector adds support for cluster table types in FROM join statements. The logic is currently limited to the joins of 2 tables. Document fields and where condition fields should be used from the leftmost table in the join condition.

      The change document date / time extraction logic is not yet supported when using joins with cluster tables.

    4. Document num. field - field that identifies the document number. If the document is defined by multiple fields, they can be listed here separated by ‘,' e.g. 'BUKRS,BELNR,GJAHR’.

      The SAP Client key field (usually MANDT) is handled automatically. The BiQ Collector default implementation selects data from the logon Client only.

    5. Date and Time field - fields in the table that can be used to select the documents relevant for the given time frame.

    6. (Optional) Document timestamp field - only available in the new UI as of release 23.5.0. Timestamp field that can be used to select the documents relevant for given time frame.

    7. (Optional) Update Date and Update Time fields - fields in the table that can be also used to identify updated records in the case that the table’s creation date/time fields are not changed on UPDATE as well.

    8. (Optional) Document update timestamp field - only available in the new UI as of release 23.5.0. Timestamp field in the table that can be also used to identify updated records in the case that the table’s creation date/time or timestamp fields are not changed on UPDATE as well.

    9. (Optional) Document and event time zone - only available in new UI as of release 23.5.0. This field can be used to override the time zone of the monitoring user to make sure that date / time values are correctly converted into event timestamps that need to be reported into analytics in UTC time zone.

    10. (Optional) Reference document number field and Reference document type - used to specify the connection between the documents if needed. In the case on the example screenshot, the Delivery document has it’s Sales order reference in the field VGBEL of a table ZLIPS. This information is used to pair the individual documents captured for the Steps into the complete document flow. In the resulting Analytics schema, all the documents that are related will have the same Global ID. Also here, multiple fields can be used, separated by ','

    11. (Optional) Reference System ID is relevant only for cross-system setup; if the previous BP step is expected to happen and be captured on the different system. If that is the case, this is the place to enter the remote system ID

    12. (Optional) Where condition - used in the select against the table/view to filter the entries. The where condition can be up to 255 characters long and the following placeholders can be used in the WHERE condition text:

      1. <SID> - SAP system ID

      2. <DATE_FROM> - Date of the previous collector execution

      3. <DATE_TO>, <DATE_NOW> - Date of the current collector execution

      4. <TIME_FROM> - Time of the previous collector execution

      5. <TIME_TO>, <TIME_NOW> - Time of the current collector execution

      6. <DATE_THIS_MONTH> - Date of the first day of the current month

      7. <DATE_THIS_YEAR> - Date of the first day of the current year

      8. <DATE_DAY_AGO> - Yesterday

      9. <DATE_WEEK_AGO> - Date of 7 days back

      10. <DATE_MONTH_AGO> - Date of 30 days back

      11. <DATE_YEAR_AGO> - Date 365 days back

      12. <MANDT> - Current logon Client

    13. Multiple rows per document - by default, the 1 schema entry per document is enforced. However, if Multiple rows per document is checked, the implementation will not merge the rows (if multiple rows are selected). This is useful for e.g. Sales orders where we want to see also the individual items (Products) in the resulting schema as separated rows

    14. Allow standalone - by default, in case the document definition has the reference document (predecessor) defined, document entries without the already captured predecessor document are thrown away. (e.g. Delivery document without already captured Sales order is ignored). “Allow standalone” checkbox overrides this behavior and allows such a document to be captured even without its predecessor. This checkbox is relevant only for documents with Reference document number field and reference document type defined.

    15. Timeless document - if checked, the date/time (and update date/time) fields are ignored, the implementation is applying only the given WHERE condition (if any). This is useful in cases when the entries are being deleted from the given table during the BP flow.

    16. Change document Object class - if filled, the document’s Date and Time fields are ignored (can be empty) and the document’s creation/update date and time are determined using the change documents of the given change document object class (CDHDR-OBJECTCLAS). This is useful when capturing the documents from SAP tables that do not have a useful date/time fields, e.g. EKKO (Purchase order header). The main document table (the one visible in Document table column, in the case the freeform JOIN, this is the first table mentioned) must have the same key fields as the respective change document entries (CDHDR-OBJECTID).

  8. Press the Save button or hit Ctrl+S.

  9. Configure the custom fields to be extracted from the document table/view:

    1. Analytics field - Technical name of the field that appears in the Analytics Schema.

    2. SAP Document field - Field in the SAP Table or View defined earlier. The type of the analytics schema field is determined based on the actual SAP table field type, except for the case of Concatenate aggregation function, where it is automatically turned to String. 

    3. Value Expression - optional expression / formula that can be used to convert the value coming from the database table field.

      1. Single expression can be up to 130 characters long

      2. Expressions can use

        • Built-in function names (case-insensitive)

        • Document table field names (case-insensitive)

        • Numerical constants without spaces or thousands separators, e.g. 100, 10000

        • Text constants, written in double-quotes, e.g. "This is a text constant"

      3. Function calls are written by specifying the function name and the parameters in brackets, separated by the coma ',' character, e.g. FUNCTION( PARAM1, PARAM2, PARAM3 )

      4. Function calls can be nested, e.g. F1( F2 ( P1, P2 ), P3 )

      5. Built-in functions

        • CONCAT( value1, value2, ... ) - Concatenates given parameters into a single value. Expects at least 2 parameters.

        • CURR_CONV ( source_currency, target_currency, amount ) - Converts the currency using the standard SAP Function module CONVERT_TO_LOCAL_CURRENCY with the current date.

        • ADD( value1, value2 ) - Adds value1 and value2 and returns the sum. If any of the parameter values cannot be converted to a number, the result will be '0' or '' according to the result field type.

        • SUB( value1, value2 ) - Subtracts value2 from value1 and returns the result. If any of the parameter values cannot be converted to a number, the result will be '0' or '' according to the result field type.

        • MUL( value1, value2 ) - Multiplies value1 and value2 and returns the result. If any of the parameter values cannot be converted to a number, the result will be '0' or '' according to the result field type.

        • DIV( value1, value2 ) - Divides value1 by value2 and returns the result. If any of the parameter values cannot be converted to a number or division by zero occurs, the result will be '0' or '' according to the result field type.

      6. The full built-in documentation can be invoked by clicking the Value expression field and pressing F1

    4. Aggregate function - Applied only if the multiple rows per document option in the document definition is not set. In such a case, the implementation:

      1. Merges the rows with the same document number 

      2. Date/time field is “max-ed” i.e. the latest row prevails

      3. Fields that do have Aggregate functions are aggregated accordingly. 

        • No aggregation - Value from the latest row is used, other values are discarded.

        • Sum, Average, Min, Max - Works for numeric fields only.

        • Concatenate, separated by ',' - Field with this aggregate function is automatically considered to be of String type.

    5. Use default conversion exit - Instructs the default step implementation to execute the default ABAP conversion exit, if defined for the data element of the given source table field. If this is checked, the target field type in the schema is converted to string automatically.

    6. Active flag - Only active setup entries are taken into account.

  10. Click the Save button or hit Ctrl+S.

The older version (20.8.0) of BiQ Collector uses the standard SAP “View cluster” for Table based BP step configuration, but has the same configuration options except:

  • Document number field and Reference document number field accept only single value.

  • Update Date field and Update Time field are missing.

  • Placeholders in the WHERE conditions for document steps.

  • Freeform SQL join clause.
  • Value extraction formulas.

Advanced Business Process Steps

More complex cases (e.g. lookup against multiple tables that are not doable via ABAP database views, or more complex monitoring logic) need to be implemented by developing a custom ABAP class implementation. This class must implement ABAP OO Interface /DVD/APPD_BIQ_IF_STEP and it’s 4 methods:

  • INIT - Gives the implementation a chance to initialize.

  • CHECK - The implementation is expected to raise exceptions of type /DVD/CX_APPD_BIQ_ERRO or it’s subclass in case of any issues found.

  • GET_STRUCTURE - Returns the list of the fields that are collected for this step.

  • COLLECT_STEP - The following parameters are provided:

    • IV_PREVIOUS_TIMESTAMP (Importing) - previous execution timestamp

    • IV_CURRENT_TIMESTAMP (importing) current timestamp of the monitoring

    • CT_DATA (Changing) - An internal table that propagates to the analytics schema. The implementation is expected to fill the custom fields that it returned using the method GET_STRUCTURE.

Once implemented, the custom ABAP class needs to be added into BiQ Collector configuration:

  1. Navigate to transaction /DVD/APPD_BIQ_SETUP.

  2. Click Business process maintenance button.

  3. Chose the BP you want to add a step for (if there is more than one BP) and double click Business process step on the left tree view.

  4. Click New entries button and fill the information accordingly, using the custom ABAP class name.

  5. Save changes (Ctrl+S or the toolbar button).

New Business Process Setup UI

As of release 22.5.0, a new version of simplified UI is introduced to enable the setup of complete processes with different step implementations (table, IDoc, etc.). Transaction /DVD/APPD_BIQ_SETUP will now display a new configuration option: Business process & steps maintenance. Original Business process maintenance and Table-based business process maintenance are now considered obsolete.

Old screens are still available but do not support selective check and selective test run. Setup is also restricted to table-based steps only.

New Main Screen for Business Process Setup

The new maintenance screen allows users to set up all of the business processes (BP) and business steps (BS) at the same time on the same screen. Section 1 contains information about all BP and corresponding BS in a hierarchical tree representation. Section 2 is reserved for detailed information and setup of corresponding BP and BS. Section 3 contains selection fields to be displayed in the analytic schema.

Application Toolbar Functions

The toolbar provides four functions.

  1. Simple refresh of Hierarchical tree.

  2. Creates new BP node in the tree.

  3. Checks the selected element on the tree. If BS is selected, then only selected BS is checked. If BP is selected and displayed, then the whole BP including its active BS is checked.

  4. Test run the selected BS or BP. If BP is selected, then all corresponding BS are part of that run too.

Tree Functions

Right-click the menu for the Business Process (BP):

  • Delete business process with steps: deletes all business steps definition one by one belonging to the selected BP. Then deletes the BP setup itself

  • Create new BP step: creates new BS into the selected BP

  • Activate processes and all the steps: checks the active flag for BP itself and all its corresponding BS

  • Deactivate process and all the steps: unchecks active flag for BP itself and all its corresponding BS

Right-click the menu for the Business Process Step (BS):

  • Delete business step: deletes all information about the Business step

Document fields

The table at the right-bottom of the page displays fields that will be collected and added to the Analytic schema sap_biq_documents. Each field definition consists of one row and must belong to a specific BS. Trg. Field is the target field name in the Analytics schema. SAP Document table field represents SAP data source. Every step type has its own definition. Table-based consist of field names from the collecting table. IDOC-based consists of the whole segment definition along with the field name at the end. The rest of the fields have the same meaning as the previous solution with one difference: Value Exp. is not relevant for the IDoc-based BS.

Business Process (BP) detail

Double clicking on any BP on the tree displays its details on the right side. Records provided in the table below display the summary of all fields defined in individual steps belonging to the particular business process.

Business Step (BS) detail

Double clicking on any BS in the tree displays its details on the right side. Both, the table-based step and IDoc-based step consist of 3 tabs:

1. Business step (general info)
2. Configuration

Document type is a logical name unique within the Business process. Used for references for upcoming steps.

Configuration logic for table-based steps remains the same as in the old UI. Only difference is the new timestamp and time zone fields added in release 23.5.0. These fields are only available in the new UI.

Configuration logic for IDoc-based step works like this:

  • Document type: Logical name. Must be unique within the Business process. Used for references for the following steps.

  • IDOC type: Type of IDocs that you want to collect (F4 help is available)

  • IDOC segment fields: Valid segment field definition consists of segment path and segment field.

    • You can specify multiple Segment field definitions and separate them with ',' character

3. SQL Editor

Table-based step SQL Editor

IDoc-based step SQL Editor

Managing Default Content

As of release 20.11.0, the AppDynamics for SAP BiQ collector comes with default out-of-box content, starting with SAP ECC Order-to-cash. The content is extended with SAP ECC Procure-to-pay process. As of release 23.5.0, the new S/4 HANA optimized and added the order-to-cash and procure-to-pay content. To manage the content:

  1. Navigate to t-code /DVD/APPD_BIQ_SETUP and click Manage default content button.
  2. The next screen shows a list of default business processes, their latest available versions, and the versions already installed in the system.

    Here you can install or update the latest available Business process by locating the required BP row and clicking the Install button.
  3. A pop-up appears asking the user to enter a name for the business process that appears in the configuration screens.
  4. BiQ Content manager shows the result logs.

Cross-System Monitoring Setup

The steps of the Business processes can be captured on multiple systems when configured.

Once the steps are captured:

  • The Business Process (BP) steps are captured in the SAP system.

  • One of the SAP systems maintain the GUIDs of the documents created in multiple systems. Any SAP system with a working SAP BiQ Installation can take the role of the Master system.

  • When the new document is captured, the SAP system checks with the Master system over the RFC for the consolidated GUID. This mechanism ensures that the BPs, even if captured by different SAP systems, are shown as belonging together in the AppDynamics Analytics.

  • The local GUIDs are also stored locally on the SAP systems. This way, only the new GUIDs are communicated via the RFC to the Master system.


The cross-system monitoring setup consists of the following actions:

  • One of the systems with the AppDynamics for SAP BiQ Collector installed and fully working needs to take the role of the Master system.

  • (ABAP) RFC connections from all other SAP Systems with BiQ installation to the Master system are used in background processing (BiQ monitoring), so they must not prompt the user to log on.

  • On every SAP System that needs to request GUIDs from the Master system, set the RFC destination in the Local setup (see AppDynamics for SAP BiQ Local setup).

  • If the BP steps are monitored using default implementation, set its Reference system ( e.g. BP Step 3 on the diagram above will set up its Reference system to “System A“).


S/4 HANA Optimized Content

New content is optimized for S/4 HANA systems so that you can use several time-range filtering methods at once along with Change Document Objects. If change documents are not recorded on your system for specific tables, you can disable the time-range filtering. To do so, clear the Change Document Object Class field on the affected business process step level.

The BIQ collector uses the fall-back filtering methods like time+date fields or the newly added timestamp fields defined on specific business process step level.