Analytics provides built-in support for collecting analytics data from various types of sources, such as agent-instrumented Java applications, .NET applications, browser applications, and more. The Analytics Events API supplements built-in analytics data sources with your custom data sources and event types. 

In addition to the transaction events published by the app agents, custom events collected by the Analytics Events API are metered. 

About the Analytics Events API

In Analytics, an event encapsulates a unit of analytics data. In APM, for example, each event corresponds to a method or service invocation, whether it be an entry point service or downstream service.

With the Analytics API, you define the structure of your own custom event in the data store, capture the event records as they occur in your custom source, and send them to the Events Service, the data store for Analytics. Once your data is in the Events Store, users can query your data through the Controller UI or the Analytics Events API.

Analytics Events API uses a shared key to authenticate clients to the Events Service. As a Controller or Analytics Administrator, you can generate API keys from the Controller UI. See Manage API Keys.

Addressing the Events Service Data Store

Unlike most REST APIs, which are presented at the Controller, you access the Analytics Events API by addressing the Events Service instance in the platform.

You address the Events Service at one of the following URLs:

You can get the Analytics Domain URLs from the SaaS Domains and IP Ranges page.

For an on-premises Events Service, address the Events Service instance host (or more likely, the virtual IP presented by a load balancer for the Events Service cluster). Use the primary default listening port for the Events Service, 9080. 

Calls to the Analytics Events API need to specify the <global_account_name>  or the <eum_account_name>for the Controller account being address, and the <api_key> generated by the administrator for this client. The API expects the values as headers, property-value pairs that are separated by a colon. As cURL arguments, for example, the values would be passed with curl through the `-H` or `--header option` as follows:

-H"X-Events-API-AccountName:<global_account_name> OR <eum_account_name>" -H"X-Events-API-Key:<api_key>"

You can get the global account name to use from the License page in the Controller UI. The API keys are described in Manage API Keys. 

The content type, also as a cURL argument, is:

-H"Content-type: application/vnd.appd.events+json;v=2" 

strongly recommends the use of SSL/HTTPS to access the API. Otherwise, the API key is sent in plain text.

Data Format

The Analytics Events API takes events as JSON-formatted name-value pair data.

Before sending data that conforms to a custom events schema, you need to define the data structure for the custom schema. The Events Service matches incoming events data to the appropriate schema.

Supported Data Types

• Boolean
• Date – Supported time formats include:
  • ISO 8601 format: yyyy-MM-dd'T'HH:mm:ss.SSSZZ.
  • UNIX epoch date format: A 13-digit number representing the number of seconds/milliseconds since UNIX epoch time (Jan 1, 1970). For example, (GMT): Mon, 17 Apr 2017 23:46:22 GMT would be 1492472782000.
• Float
• Integer
• String

Naming Restrictions

Custom event names and field names must conform to the following:

• Contain only a-z, A-Z, _ (underscore), 0-9.
• Names can not start with a number.

Timestamp Fields

Two implicit timestamp fields are automatically added to custom schemas:

• eventTimestamp
• pickupTimestamp

The eventTimestamp field represents the time an event occurred. An API client can specify a value for the timestamp field when it creates an event. If it does not, the Analytics Events API uses the same value for eventTimestamp as it uses for another implicit field, pickupTimestamp.The pickupTimestamp field, which is always populated by the Events Service, represents the time the event was received by the Events Service.

When configuring a milestone in Business Journeys, you need to supply a date in the eventTimestamp field in order for the event to register. In order to use Custom Events to define Business Journey, the custom events should send the eventTimestamp field. If not, the Business Journeys will not work.

uses another timestamp, eventcompletionTimestamp, which is not automatically added to custom schemas. The eventcompletionTimestamp field represents the time an event ends. Generally, uses this field internally to accurately report long-running events. 

You can express all timestamp fields using ISO 8601 or UNIX epoch time (64-bit milliseconds) format.

Example API Call Flow

The following steps take you through an on-premises API call workflow for using the Analytics Events API. The steps show cURL examples for creating a schema, publishing an event to that schema, and then querying the event. 

1. Define the schema by associating field names with data types. For example, the following defines a Purchase event type: 

   curl -X POST "<events_service_endpoint>:9080/events/schema/myProducts" -H"X-Events-API-AccountName:customer1_1234-567a-bccc-123" -H"X-Events-API-Key:a123b456-c789-1d23-e456-nnn" -H"Content-type: application/vnd.appd.events+json;v=2" -d '{"schema" : { "id": "string", "productBrand": "string", "userRating": "integer", "price": "float", "productName": "string", "description": "string" } }'

2. Publish an event based on the schema you created: 

   curl -X POST "<events_service_endpoint>:9080/events/publish/myProducts" -H"X-Events-API-AccountName:customer1_1234-567a-bccc-123" -H"X-Events-API-Key:a123b456-c789-1d23-e456-nnn" -H"Content-type: application/vnd.appd.events+json;v=2" -d '[{"id": "5653b879ab33a","productBrand": "ACME","userRating": 3,"price": 2006.41,"productName": "Watch","description": "new watch"},{"id": "5653b879700","productBrand": "Widget","userRating": 1,"price": 3800.13,"productName": "Watch","description": "2015 watch"}]'

3. Query the event data:

   curl -X POST "http://<events_service_endpoint>:9080/events/query" -H"X-Events-API-AccountName:customer1_7xxx-467a-bccc-xxx" -H"X-Events-API-Key:a123b456-c789-1d23-e456-nnn" -H"Content-type: application/vnd.appd.events+text;v=2" -d 'SELECT * FROM myProducts'

If including fields with ADQL keywords, enclose the keywords in single quotes. These keywords include, for example, between, in, select, and others. 

For a single query request, use this content type:

-H"Content-type: application/vnd.appd.events+text;v=2" 

In a multi-query request, the queries are passed as JSON body text. In this case, use the following content type header:

-H"Content-type: application/vnd.appd.events+json;v=2"

Custom Event Ingestion Limits

Controller ingestion of custom events has the following limits:

• Fields: 255 maximum per event type
• String attributes: 4 kb maximum length
• Batch total count: 10,000 events per call
• Batch total size: 5 Mb maximum per call
• Max custom event schemas for an account: 20

Custom Events Limit Increase 

By default, the Events Service allows you to create up to 1500 custom fields. If you reach 80% capacity of custom fields (1200 fields), the maximum field limit increases by 500 fields at the next index rollover. 

For example, if you reach 1400 custom fields, your maximum field limit will increase from 1500 to 2000 fields. If you later reach 1800 custom fields, and maximum increases to 2500 fields. 

You can create an absolute maximum of 3000 custom fields. Reaching or exceeding the maximum fields could result in lag or outages. 

Publish Events

The Publish Events API call takes an array of events and stores them in the Events Service storage. The data must comply with an existing schema. A single request cannot publish to multiple event types. 

If the event data doesn't match an event schema, the Events Service makes a best-effort attempt to match the data to the schema and returns a 400 bad request if unsuccessful. 

Format

POST https://analytics.api.example.com/events/publish/{schemaName}

POST http://<events_service_endpoint>:9080/events/publish/{schemaName}

Query Params

N/A

Path Parameters

Headers

Example SaaS Publish 

POST https://analytics.api.example.com/events/publish/{schemaName} 
X-Events-API-AccountName:<global_account_name> OR <eum_account_name>
X-Events-API-Key:<api_key> 
Content-Type: application/vnd.appd.events+json;v=2 Accept: application/vnd.appd.events+json;v=2
[
   {
      "id": "5653b879ab33a",
      "productBrand": "ACME",
      "userRating": 3,
      "price": 2006.41,
      "productName": "Watch",
      "description": "new watch"
   },
   {
      "id": "5653b879700",
      "productBrand": "Widget",
      "userRating": 1,
      "price": 3800.13,
      "productName": "Watch",
      "description": "2015 watch"
   }
]

HTTP/1.1 202 ACCEPTED

Error Codes

Create Event Schema

Use this API to create your own event schema. The schema defines the overall structure of an event type by field and type. 

You only need to use this API if the event you are uploading does not match an existing schema for first-class event types (such as logs or transactions). Events that conform to an existing schema automatically match that schema. Review the supported data types and naming restrictions described prior to this topic.

Format

POST https://analytics.api.example.com/events/schema/{schemaName}

POST http://<events_service_endpoint>:9080/events/schema/{schemaName}

Path Params

Query Params

N/A

Headers

Example SaaS Create

POST http://analytics.api.example.com/events/schema/{schemaName} HTTP/1.1
X-Events-API-AccountName:<global_account_name>
X-Events-API-Key:<api_key>
Content-Type: application/vnd.appd.events+json;v=2
Accept: application/vnd.appd.events+json;v=2
{
   "schema" : {
      "account": "integer",
      "amount": "float",
      "product": "string"
   }
}

HTTP/1.1 201 CREATED

Retrieve Event Schema

Use this API to retrieve an existing event schema.

Format

GET http://analytics.api.example.com/events/schema/{schemaName}

GET http://<events_service_endpoint>:9080/events/schema/{schemaName}

Path Params 

Query Params

N/A

Headers

Example SaaS Retrieve

GET http://analytics.api.example.com/events/schema/{schemaName} HTTP/1.1
X-Events-API-AccountName:<global_account_name> OR <eum_account_name>
X-Events-API-Key:<api_key>
Accept: application/vnd.appd.events+json;v=2

HTTP/1.1 200 OK
{
   "schema" : {
      "account": "integer",
      "amount": "float",
      "product": "string"
   }
}

Update Event Schema

Use this API to update an existing event schema by field. The request body defines the updates to be applied to the event schema.

As shown in the following example, you specify each field update action as a named section in the request body. The actions are represented by these fields:

• Add field
• Rename field

For the Add field definition, you need to specify the data format for the new field, as you would when creating the event schema.

The response to this call should be the complete event schema as modified.

Format

PATCH http://analytics.api.example.com/events/schema/{schemaName}
X-Events-API-AccountName:<global_account_name> OR <eum_account_name>
X-Events-API-Key:<api_key>

PATCH http://<events_service_endpoint>:9080/events/schema/{schemaName}

Path Parameters

Query Params

N/A

Headers

Example SaaS Update

PATCH http://analytics.api.example.com/events/schema/{schemaName} HTTP/1.1
X-Events-API-AccountName:<global_account_name> OR <eum_account_name>
X-Events-API-Key:<api_key>
Content-type: application/vnd.appd.events+json;v=2
Accept: application/vnd.appd.events+json;v=2
  
[
  {
    "add": {
      "newfield": "integer"
    },
    "rename": {
      "oldname": "newname",
      "oldname2": "newname2"
    }
  }
]

HTTP/1.1 200 OK

Delete Event Schema

Use this API to delete an existing event schema.

Format

DELETE http://<events_service_endpoint>:9080/events/schema/{schemaName}
X-Events-API-AccountName:<global_account_name> OR <eum_account_name>
X-Events-API-Key:<api_key>

DELETE http://<events_service_endpoint>:9080/events/schema/{schemaName}

Path Params

Query Params

N/A

Headers

Example SaaS Delete Request

DELETE http://analytics.api.example.com/events/schema/{schemaName} HTTP/1.1
X-Events-API-AccountName:<global_account_name> OR <eum_account_name>
X-Events-API-Key:<api_key>
Accept: application/vnd.appd.events+json;v=2

Querying Events

When querying analytics events data, the following applies:

• Every Events Service API has a limit of 200 searches per minute by each account on each event type.

• The Multi-Query Events API is limited to twenty queries per HTTP request.

• The Analytics Query API can return a maximum of 10,000 results. If you would like to retrieve and paginate over the data, you can use the scroll mode. Scroll mode is limited to 1,000 results per batch.

  Scroll mode cannot be used in Query Events (Multiple Queries).

• ADQL sets the rate limits on the following header fields to control large request volumes that may impact system stability:

  • X-Events-API-AccountName: 450 requests per minute.
  • X-Events-API-Key: 145 requests per minute.
• Limits work differently for aggregation and non-aggregation queries. Because we use the limits specified in the ADQL query as the bucket count limit, it is not possible to also use it as the overall result count limit. Therefore, the URL parametric limit is used for the overall limit. In the case of non-aggregation queries, there is no bucket limit, so the limit specified in the ADQL query is taken as the row count limit and the URL parametric limit becomes the second place to look for it if the ADQL query does not specify a limit.
  • Aggregation queries: the total returned row count is limited by the limit in the URL query parameter, and is not directly related to the limits specified in the ADQL query statement itself. The limits in the ADQL query apply only to bucket counts in aggregations.
  • Non-aggregation queries: If LIMIT is not specified in the SELECT statement, the value specified in the URL query parameter is used. If the limit query parameter is also absent, the default is 100.

Query Events (Single Query)

This API can be used either as a simple text-formatted query or as a JSON-formatted query. The JSON-formatted query can accommodate multiple queries per call and is described in Query Events (Multiple Queries). 

An event type might search against multiple event types. Therefore, the event type is not provided in the URL path or as a query parameter, but as part of the ADQL query provided in the request body itself. Your ADQL queries must adhere to the syntax described in the ADQL Reference.

This section describes the single query form for querying events. 

Format

POST http://analytics.api.example.com/events/query?limit=20

POST http://<events_service_endpoint>:9080/events/query
X-Events-API-AccountName:<global_account_name> OR <eum_account_name>
X-Events-API-Key:<api_key>
Content-type: application/vnd.appd.events+text;v=2

Query Params

Headers

Example SaaS Query

POST http://analytics.api.example.com/events/query?start=1422823420000&end=1423687476000&limit=20000 HTTP/1.1
X-Events-API-AccountName:<global_account_name> OR <eum_account_name>
X-Events-API-Key:<api_key>
Content-Type: application/vnd.appd.events+text;v=2
Accept: application/vnd.appd.events+json;v=2
 
SELECT * FROM county WHERE size>=30 AND population>20000

HTTP/1.1 200 OK
{
    "total": 10000,
    "fields": [
      { "label": "eventTimestamp",  "field": "eventTimestamp",  "type": "date",    "aggregation": null },
      { "label": "size",             "field": "size",             "type": "integer", "aggregation": null },
      { "label": "population",          "field": "population",          "type": "integer", "aggregation": null },
      { "label": "pickupTimestamp", "field": "pickupTimestamp", "type": "date",    "aggregation": null }
    ],
    "results": [
      [ "2015-01-03T23:55:39.801-08:00", 35, 47500, "2015-02-11T19:52:28.805Z" ],
      ...
    ]
}

curl -s -X POST "<events_service_endpoint>:<events_service_port>/events/query?start=1422823420000&end=1423687476000" \
-H"X-Events-API-Key:<api_key>" \
-H"X-Events-API-AccountName:<global_account_name> OR <eum_account_name>" \
-H"Content-Type:application/vnd.appd.events+text;v=2" \
-H"Accept:application/vnd.appd.events+json;v=2" \
-d"[{
  \"query\": \"SELECT eventTimestamp, pagename FROM browser_records ORDER BY eventTimestamp DESC\",
  \"mode\": \"scroll\"
  }]"

[{"label":"0","fields":[{"label":"agentid".........,[],[],[],[],[]]],"moreData":true,"scrollid":"MCwxMDAwMDAsMTAwMCxEbkYxWlhKNVZHaGxia1psZEdOb0JBQUFBQUFERW1pbUZqZ3RVakpNUkRKVFVVSkxVbkU1ZEROVlpqRkNhVUVBQUFBQUExQkF6QlpJY1hCM1dqVTNWbEp6UTBOaUxUZHlRbkJJZVZoM0FBQUFBQU11Q1dJV1N6VlpUbFZvV1Y5Uk15MUhSbXRSVTFScWRVWkNkd0FBQUFBQzF4TTZGbGRXYW1FMU5HNUVVazk1WVdaUWVXWlRlSEZLZFZFPQ==","schema":"browserrecord"}]

curl -s -X POST "<events_service_endpoint>:<events_service_port>/events/query?start=1422823420000&end=1423687476000" \
-H"X-Events-API-Key:<api_key>" \
-H"X-Events-API-AccountName:<global_account_name> OR <eum_account_name>" \
-H"Content-Type:application/vnd.appd.events+text;v=2" \
-H"Accept:application/vnd.appd.events+json;v=2" \
-d"[{
  \"query\": \"SELECT eventTimestamp, pagename FROM browser_records ORDER BY eventTimestamp DESC\",
  \"mode\": \"scroll\",
     \"scrollid\":\"MCwxMDAwMDAsMTAwMCxEbkYxWlhKNVZHaGxia1psZEdOb0JBQUFBQUFERW1pbUZqZ3RVakpNUkRKVFVVSkxVbkU1ZEROVlpqRkNhVUVBQUFBQUExQkF6QlpJY1hCM1dqVTNWbEp6UTBOaUxUZHlRbkJJZVZoM0FBQUFBQU11Q1dJV1N6VlpUbFZvV1Y5Uk15MUhSbXRSVTFScWRVWkNkd0FBQUFBQzF4TTZGbGRXYW1FMU5HNUVVazk1WVdaUWVXWlRlSEZLZFZFPQ==\" 
  }]"

Query Events (Multiple Queries)

Use this API to execute multiple queries against a particular account and event type in parallel. A query event that specifies multiple queries does so by including multiple ADQL queries in the body of the request. Your ADQL queries must adhere to the syntax described in the ADQL Reference.

Using queries in this form takes advantage of certain backend optimizations in query performance. Query filter criteria, such as time range and limit, can be overridden by each inner query.

Format

POST http://analytics.api.example.com/events/query?limit=20

POST http://<events_service_endpoint>:9080/events/query
X-Events-API-AccountName:<global_account_name> OR <eum_account_name>
X-Events-API-Key:<api_key> 

Path Params

None

Query Params

Headers

Payload

Example SaaS Multiple Query

POST http://analytics.api.example.com/events/query?limit=100 HTTP/1.1
X-Events-API-AccountName:<global_account_name> OR <eum_account_name>
X-Events-API-Key:<api_key>
Content-Type: application/vnd.appd.events+json;v=2
Accept: application/vnd.appd.events+json;v=2
[
    {
      "label": "high_population",
      "query": "SELECT * FROM county WHERE population>50000",
      "limit": 10,
      "start": "2017-02-23T0:0:0Z",
      "end": "2017-03-1T0:0:0Z"
    },
    {
      "label": "small_area",
      "query": "SELECT * FROM county WHERE size<25",
      "start": "2017-02-23T0:0:0Z",
      "end": "2017-03-1T0:0:0Z"
    },
    {
      "label": "high_population_density",
      "query": "SELECT * FROM county WHERE population>50000 AND size<25",
      "limit": 100,
      "start": "2017-02-23T0:0:0Z",
      "end": "2017-03-1T0:0:0Z"
    }
]

HTTP/1.1 200 OK
[
  {
    "label": "high_population",
    "total": 30,
    "fields": [ ... ],
    "results": [ ... ]
  },
  {
    "label": "small_area",
    "total": 50,
    "fields": [ ... ],
    "results": [ ... ]
  },
  {
    "label": "high_population_density",
    "total": 10,
    "fields": [ ... ],
    "results": [ ... ]
  }
]