Filter Sensitive Data Using the Collector

This document contains references to OpenTelemetry™ documentation. AppDynamics does not own any rights and assumes no responsibility for the accuracy or completeness of such third-party documentation.

While sensitive data can be helpful for diagnosis and troubleshooting, security considerations may require filtering certain information. Data filtering can be done at the agent level or the Collector level. This page includes the recommended configurations to filter the sensitive data at the Collector level. Sensitive data such as identity, credit card information, or email addresses require special data management to ensure data protection and compliance with industry standards.

Sensitive data sent to AppDynamics can be removed using the AppDynamics Distribution for OpenTelemetry Collector or the OpenTelemetry Collector. To filter the sensitive data using the Collector, perform the following steps:

Add the required processors to the Collector's configuration file. The processors allow you to delete, redact, or hash specific attributes.
Activate the processor functionality by modifying the appropriate service | pipelines.

The processors available for filtering sensitive data are:

attributes
redaction
transform

Common Filtering Scenarios

Attributes

Use the attributes processor to remove specific keys such as credit card information, passwords, and other sensitive values. See Attributes Processor and General Identity Attributes.

Credit Card Information

You can redact values that appear in the credit_card_number attribute:

...
processors:
    attributes/update:
        actions:
            - key: credit_card_number
              value: redacted
              action: update
...
service:
    pipelines:
        traces:
            processors: [..., attributes/update, ...]
        metrics:
            processors: [..., attributes/update, ...]
        logs:
            processors: [..., attributes/update, ...]
...

CODE

Passwords

You can delete a user.password:

...
processors:
    attributes/update:
        actions:
            - key: user.password
              action: delete
...
service:
    pipelines:
        traces:
            processors: [..., attributes/update, ...]
		metrics:
			processors: [..., attributes/update, ...]
		logs:
		 	processors: [..., attributes/update, ...]
 ...

CODE

Redaction

Use the redaction processor to remove all attributes except description, group, id, and name. See Redaction Processor.

... 
processors:
  redaction/update:
    allow_all_keys: false
    allowed_keys:
      - description
      - group
      - id
      - name
...
service:
    pipelines:
        traces:
            processors: [..., redaction/update, ...]
        metrics:
            processors: [..., redaction/update, ...]
        logs:
            processors: [..., redaction/update, ...]
 
  ...

CODE

The redaction processor includes a summary configuration that provides details on what is filtered.

Use the redaction processor to block the card numbers using regular expression filters for Visa and Master card formats.

... 
processors:
  redaction/update:
    allow_all_keys: true
    blocked_values:
      - "4[0-9]{12}(?:[0-9]{3})?" ## Visa credit card number
      - "(5[1-5][0-9]{14})"       ## MasterCard number
    summary: debug 
...
service:
    pipelines:
        traces:
            processors: [..., redaction/update, ...]
        metrics:
            processors: [..., redaction/update, ...]
        logs:
            processors: [..., redaction/update, ...]
...

CODE

Transform

Use the transform processor to mask specific values. See Transform Processor.

Create Allowlist for Specific Keys

Use the transform processor to remove all attributes except service.name, service.namespace, cloud.region, process.command_line:

...
processors:
    transform/update:
        traces:
            queries:
               - keep_keys(resource.attributes, "service.name", "service.namespace", "cloud.region", "process.command_line")
        metrics:
            queries:
               - keep_keys(resource.attributes, "service.name", "service.namespace", "cloud.region", "process.command_line")
        logs:
            queries:
               - keep_keys(resource.attributes, "service.name", "service.namespace", "cloud.region", "process.command_line")

  ...
service:
    pipelines:
        traces:
            processors: [..., transform/update, ...] 
        metrics:
            processors: [..., transform/update, ...] 
        logs:
            processors: [..., transform/update, ...]
...

CODE

Mask a Partial Value for a Specific Key

Use the transform processor to redact a password that appears on the command line. For example, $env password=mysecret username=myusername python run-my-app.py.

...
processors:
    transform/update:
        traces:
            queries:
               - replace_pattern(resource.attributes["process.command_line"], "password\\=[^\\s]*(\\s?)", "password=***")
         metrics:
            queries:
               - replace_pattern(resource.attributes["process.command_line"], "password\\=[^\\s]*(\\s?)", "password=***")
         logs:
            queries:
               - replace_pattern(resource.attributes["process.command_line"], "password\\=[^\\s]*(\\s?)", "password=***")
...
service:
    pipelines:
        traces:
            processors: [..., transform/update, ...]
        metrics:
            processors: [..., transform/update, ...]
        logs:
            processors: [..., transform/update, ...]
...

CODE

OpenTelemetry™ is a trademark of The Linux Foundation®.