You must configure the Splunk Universal Forwarder to send the application metadata from Splunk AppDynamics SaaS to Splunk Platform for application context correlation.

Supported Version

Universal Forwarder version >= 7.0.0

Before You Begin

Ensure the following:

  • You must have an universal forwarder already deployed in your environment. For more information about the deployment, see Deploy Universal Forwarder in Splunk Documentation.
  • The universal forwarder must be functioning properly and sending logs to Splunk Platform. If there are any issues, contact Splunk Support.

Configuration

To configure your universal forwarder for log observer connect for Splunk AppDynamics SaaS:

  1. In the $SPLUNK_HOME/etc/system/local/ directory, open the inputs.conf file. If the file is not available, create a file with the name inputs.conf. For specification, see inputs.conf.spec.
  2. In each monitor stanza, add the _meta field.
  3. In _meta field, add the AppDynamics application name and the tier name. The following sample displays a stanza in the inputs.conf file that collect logs from the directory /var/log/my-app:

    [monitor:///var/log/my-app]
    index = <index-name>
    disabled = 0
    _meta = appd_app_name::<APPD_APP_NAME> appd_tier_name::<APPD_TIER_NAME>
    CODE

  4. Restart your Splunk instance to enable the configuration changes.

For logs to get parsed and indexed properly with respect to the injected meta data, you must properly configure event line breaking. Multiline logs must be handled correctly so that they do not get split up into multiple events. Also, multiple logs must not add together into one event. For information on how to configure line breaking, see the Splunk Documentation.

Additional Configuration for Structured Logs

To enable log connect for structured logs, configure the following in your universal forwarder.

No Preconfigured Source Type

If you do not have any preconfigured source type for the structured logs, you need to specify the source type in the inputs.conf file:

  1. In each stanza containing the structure log file, specify the format of the structured logs. The following formats are supported:
    FormatValue
    Comma-separated valuescsv
    JSON_json
    Pipe-separated valuespsv
    Tab-seperated valuestsv

  2. Restart the universal forwarder.

For example, if your structured log is in JSON format, specify the sourcetype as _json in the stanze:

[monitor:///var/log/my-app]
index = <index-name>
sourcetype=_json
disabled = 0
_meta = appd_app_name::<APPD_APP_NAME> appd_tier_name::<APPD_TIER_NAME>
CODE

Preconfigured Source Type

If you have already configured a custom source type for the structured logs but indexed field extraction is not enabled, you need to update the props.conf in your universal forwarder and select the appropriate index extraction on your Splunk instance.

In the universal forwarder:

  1. Go to $SPLUNK_HOME/etc/system/local/props.conf.
  2. Update the stanza to enable INDEXED_EXTRACTIONS by specifying one of the following formats:
    • CSV
    • JSON
    • TSV
    • PSV
    • W3C
  3. Restart the universal forwarder.

For example, if the custom source type is custom-json, update the stanza as follows:

[custom-json]
INDEXED_EXTRACTIONS = JSON
CODE

In the Splunk instance:

  1. Click Settings > Data> Source Types.
  2. Click New Source Type.
  3. In the Name field, specify the same custom source type name that you have mentioned on UF. For example, custom-json.
  4. In the Description field, enter an appropriate description for the source type.
  5. In the Destination app list, select the application as system.

    You can select the Destination app list when you are using Splunk Cloud Platform or Splunk Enterprise deployment on Classic Experience. For Splunk Cloud Platform on Victoria Experience, the destination app is configured to 000-self-service by default. Contact your Splunk administrator to enable object sharing for the 000-self-service app to Global.

  6. In the Category list, select the category as Structured.
  7. In the Indexed extraction list, select the format for the custom source type. The format can be one of the following:
    • CSV
    • JSON
    • TSV
    • PSV
    • W3C
  8. Click Advanced and add New setting.
  9. In the KV_MODE list, select the value as none.
  10. Click Save.

The following image shows an example source type: