Filter Sensitive Data

By default, the Java Agent sends transaction data to the Controller that your organization may classify as privileged information. Although such data is useful for diagnosis and troubleshooting, security considerations may require you to filter certain information from view in the Controller. You can use:

• Sensitive URL filters to exclude sensitive information from a URL in snapshot details. 
• Sensitive message filters to exclude sensitive data that the application may place in log messages or exception detail messages.

Data Filter Format 

Data Filters are used when setting up URLs in snapshots or analytics. 

{
  "appliesTo": "...",
  "matchPattern": "..."
}

• matchPattern(required): A regular expression defining the sensitive data that triggers the filtering.
• appliesTo (required): A tag defining on which kind of data the pattern should be applied. The possible values are env-vars, http-cookies, http-session-key, http-headers, http-params, and user-data. 
  • env-vars tags are defined by the default two data filters that contain a password or key. However, these filters can be overwritten with a valid custom filter for the environment variables. 
  • Http and user-data tags are considered only when processing data in Transaction Snapshots or in Analytics Reports.

Default Sensitive Data Filters

When you enable a sensitive data filter, the Controller displays asterisks for the values of matching environment variables or system properties. By default, the Java Agent enables two sensitive data filters by passing the configuration to the addAppAgentSensitiveDataFilters() method:

{
  "messageFilters": [
    { "messageType": "throwable",
      "matchPattern": "test" },
    { "messageType": "throwable",
      "matchPattern": "ip",
      "redactionRegex": "(\\d+\\.)+\\d+" }
  ],
  "dataFilters": [
    { "appliesTo": "http-headers",
      "matchPattern": "test" }
  ],
  "urlFilters": [
    { "delimiter": "/",
      "segment": "1,2",
      "matchPattern": "test1",
      "paramPattern": "test2" }
  ]
}

Add a Sensitive URL Filter

You can use sensitive URL filters to configure the agent to obfuscate sensitive information from the URLs in transaction snapshot details.

1. Add a sensitive URL filter element as a child of the sensitive URL filters element:

   {
     "delimiter": "...",
     "segment": "..." ,
     "matchPattern": "...",
     "paramPattern": "..."
   }

   CODE

• delimiter: 
• segment: Specify a comma-separated list to indicate the segments that you want the agent to filter. Segment numbering starts from 1. 
• match-pattern: Specify the string that you want to be filtered by the match-filter.
• param-pattern: Specify the regular expression matching the query parameters to filter.

• Specify the character that you want to use as URL segment endpoints. The agent splices the URL at each delimiter instance to create the segments. For HTTP, use the forward slash character "/".  

  

   The indexing starts from 1 and that the protocol names (http://, https://, ...) are not treated as special cases, so for URLs containing protocol: "http://myapp.example.com/sensitive/data?first_name=abc&last_name=xyz" the segments are:

  • http:
  • " "
  • myapp.example.com
  • sensitive
  • data

For example, the following configuration splits the URL on the "/" character and masks the fourth segment and the param-pattern in the third segment of the URL. In this case, the segmentation and obfuscation apply only to URLs containing "myapp".

{
  "delimiter": "/",
  "segment": "4" ,
  "matchPattern": "myapp",
  "paramPattern": "[a-z]+_name"
}

The exit call to https://myapp.example.com/sensitive/data?first_name=abc&last_name=xyz breaks down to three segments: "https://myapp.example.com/", "sensitive", and "data?first_name=abc&last_name=xyz". The Controller shows the masked values of the URL and the param-pattern display https://myapp.example.com/*****/data?first_name=***&last_name=*** in the snapshot details.

Add a Sensitive Message Filter

You can use sensitive message filters to configure the agent to obfuscate sensitive information contained within text messages collected by the agent from log messages, or detail messages from exceptions. 

1. Add a sensitive message filter element as a child of the sensitive message filters element:

{   
  "messageType": "",        // type of message for which the filter applies
  "matchPattern": "",       // regex filter to search in data
  "redactionRegex": ""      // regex filter of how masking is applies(optional), by default masks entire data
}

• messageType specify throwable

• matchPattern specify the pattern that, when matched, opts the message in for redaction

• redactionRegex specify a regular expression identifying data that should be redacted from the opted-in messages

For example, if an application includes secret numeric values in its exception messages, the following configuration would mask any references of it:

{ 
   "messageType": "throwable", 
   "matchPattern": "test", 
   "redactionRegex": "\\d+" 
}

The exception message:`Unable to update the user profile (ID: 4215908), corrupted or incoherent data given` will be collected as `Unable to update the user profile (ID: ****), corrupted or incoherent data given.`