To configure the Log Collector, add log-specific settings to the same values YAML file that you used to deploy Kubernetes and App Service Monitoring. This file is typically named collectors-values.yaml. If you can't find it, see Recover a Lost Values YAML File.

There are two methods of updating collectors-values.yaml. You can generate a basic configuration or you can update the file yourself. Both of the following methods are explained:

Generate a Basic Configuration

You can generate a basic collectors-values.yaml to simplify the deployment of the Log Collector onto your clusters. 

  1. From the Cisco Cloud Observability Tenant UI, navigate to Configure > Kubernetes and APM.
  2. Enter your Credential set name and Kubernetes cluster name.
  3. In the Enable Additional Configurations section, select the Log Collector Agent checkbox, and select which operating system(s) you need special settings for.
  4. Click Generate configuration file.
    This generates a basic collectors-values.yaml file for you, which simplifies the deployment of the Log Collector onto your clusters. 
  5. You can either copy the code snippet (minimal) or you can download the generated file, which has more comments and settings that are commented-out. 
  6. Update your collectors-values.yaml using either of these options:
    • If you have downloaded the file and you need to un-comment additional settings in it, see Add Your Own Configuration.
    • If you have copied a snippet, overwrite that section of your existing collectors-values.yaml.
  7. See Next Steps.

Add Your Own Configuration

Whether or not you generate a basic collectors-values.yaml, you can update it with the settings you need for your Log Collector deployment as follows. The following steps omit most optional settings. For a complete description of all settings, valid values, and so on, preview Log Collector Settings before completing the following steps.

Set Required Parameters in global

  1. On Cisco Cloud Observability, get the name of your cluster:

    1. On the Observe page, select your cluster.
    2. In the Properties panel, copy the name of your cluster and save it to a text file.

  2. In global, set clusterName to the name of your cluster. This name must match the cluster's name as seen in Cisco Cloud Observability:

    global:
  clusterName: <cluster-name>
    YML

Set Required Parameters in appdynamics-cloud-k8s-monitoring.install

In appdynamics-cloud-k8s-monitoring.install, set logCollector to true: 

install:
  ...   
  logCollector: true
YML

Set Optional Parameters in appdynamics-cloud-k8s-monitoring.logCollectorPod

In appdynamics-cloud-k8s-monitoring.logCollectorPod , set rollingUpdateMaxUnavailable to specify the maximum number of Log Collector pods that can be unavailable during an update. You can specify this value as a number or as a percentage. For percentage, specify a value between 0% (wait for each pod to restart) and 100% (don't wait for any pods to restart). On new deployments, this parameter is already set to 100%, but on existing deployments, you need to add this parameter. For more information, see Log Collector Settings.
For example: 

  logCollectorPod: 
    rollingUpdateMaxUnavailable: 70%
YML

Create the appdynamics-cloud-k8s-monitoring.logCollectorConfig Key

If your collectors-values.yaml does not have the appdynamics-cloud-k8s-monitoring.logCollectorConfig key, copy and paste it from the sample in Log Collector Settings.

Set Optional Defaults in appdynamics-cloud-k8s-monitoring.logCollectorConfig.container.defaultConfig

In appdynamics-cloud-k8s-monitoring.logCollectorConfig.container.defaultConfig , set parameters to your default log harvesting conditions. You only need to specify overriding values. In other words, if you want to use the default value of any parameter, you can omit it. However, if you do define defaultConfig, it must include logFormat. If you don't specify logFormat, you can't mask sensitive data contained within the log messages that are ingested through this configuration.

For new deployments, you don't need to change anything in collectors-values.yaml for default log collection; simply deploy the Log Collector and see your logs in the system immediately, parsed by timestamp only. You can then incrementally refine your parsing configurations using the parsing pattern tester (see Log Parsing Validator). If you don't want to enable default log collection, set defaultConfig.enabled to false.

For example: 

  logCollectorConfig: 
    os: [linux, windows]                                        
    container:
      defaultConfig: 
        enabled: true;           
        multiLinePattern: '^{'
        multiLineMatch: "after"
        multiLineNegate: true
        logFormat: "logs:email_logs"
        messageParser:           
          json:
            enabled: true
YML

Set Optional Parameters in appdynamics-cloud-k8s-monitoring.logCollectorConfig.container.conditionalConfigs

  1. In appdynamics-cloud-k8s-monitoring.logCollectorConfig.container.conditionalConfigs , create a pair of condition+config  sections for every log-generating container or Kubernetes infrastructure:

    1. In condition, specify the condition that the container must match in order for the Log Collector to collect its logs.
      A condition has 3 fields: operator (equals , contains, or, and), key (name of a property), and value (value of property which must be matched).
      For example:

        logCollectorConfig: 
    os: [linux, windows]                                        
    container:
      defaultConfig:            
        ...     
      conditionalConfigs: 
       - condition:
          or:
            - equals:
                kubernetes.container.name: log-gen-app-log4j-windows
            - equals:
                kubernetes.container.name: log-gen-app-log4j-linux
      YML

    2. In config, set one of the messageParser* parameters:

      1. If your log messages can span multiple lines, set multilinePattern, multilineNegate, and multilineMatch to the correct values to properly parse multiline log messages.

      2. Set logFormat to "<log-namespace>:<log-description>". For example, "K8s:cis_logs". For more examples, see Log Collector Settings.  If you don't specify logFormat, you can't mask sensitive data contained within the log messages that are ingested through this configuration. See Mask Sensitive Data.

      3. Set enabled to true.

      4. Set pattern to the regular expression that matches log messages of this parser type.
        For example, for multiline log4J logs:  

          logCollectorConfig:
    os: [linux, windows]                                        
    container:
      defaultConfig:            
        ...
      conditionalConfigs: 
       - condition:
          ...
         config: 
          multiLinePattern: '^2023'
          multiLineNegate: true
          multiLineMatch: "after"
          logFormat: "logs:email_logs"
          messageParser:
            log4J:
              enabled: true
              pattern: "%d{yyyy-MM-dd'T'HH:mm:ss} %p %C{1.} [%t] %m%n"
        YML

If You Are Deploying the Log Collector on Windows Containers

  1. Set appdynamics-cloud-k8s-monitoring.logCollectorConfig.os to the list of operating systems your containers are running. Valid values are linux, windows: 

      logCollectorConfig:
    os: [windows,linux]
    YML

  2. (Optional) To deploy different Log Collector images to different operating systems, set appdynamics-cloud-k8s-monitoring.logCollectorPod.env.linux.image and appdynamics-cloud-k8s-monitoring.logCollectorPod.env.windows.image: 

    logCollectorPod:
  imagePullPolicy: IfNotPresent
    env:
      linux:
        image: <image-url>
      windows:
        image: <image-url>
    YML

  3. (Optional) If you need to operating system-specific overrides for appdynamics-cloud-k8s-monitoring.logCollectorConfig.container, set appdynamics-cloud-k8s-monitoring.logCollectorConfig.env. You must override some sections of appdynamics-cloud-k8s-monitoring.logCollectorConfig.container entirely. See exact requirements in Log Collector Settings.

      logCollectorConfig:
    os: [windows, linux]
    env: 
      linux: 
        container:
          defaultConfig:
                ...
          conditionalConfigs:
            - condition:
                ...
             config:
                ...
          logging:
            ...
          monitoring:
            ...         
      windows:    
        container:
          defaultConfig:
                ...
          conditionalConfigs:
            - condition:
                ...
             config:
                ...
          logging:
            ...
          monitoring:
            ...
    YML

Next Steps

  1. Validate collectors-values.yaml with a YAML validator like YAML Lint.
  2. To apply the new configuration to your cluster, see Apply Changes to Your Cluster in Deploy the Log Collector.

OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.