Correlate Application Logs with Infrastructure

To set up correlation between application logs and the infrastructure that the applications are running on, use the following steps that correspond to your infrastructure.

Logs from Applications Running on Kubernetes

Cisco Cloud Observability can correlate logs from Kubernetes-based applications to their underlying Kubernetes infrastructure, such as container, node, pod, workload, namespace and cluster entities:

• If you're using to Kubernetes and App Service Monitoring to send logs to Cisco Cloud Observability from Kubernetes clusters, the Log Collector embeds the container.id attribute in every log message, so Cisco Cloud Observability automatically correlates them to the related infrastructure entities.

• If you're using an OpenTelemetry collector to send logs to Cisco Cloud Observability from Kubernetes clusters, enrich your logs with the k8s.cluster.id and container.id attributes. The k8s.cluster.id attribute must have a value equal to the UUID of the kube-system namespace. See Logs Ingested Using OpenTelemetry.

Logs from Applications Running on AWS Infrastructure

Enrich your logs with the service.name and service.namespace attributes:

• If your applications are running on Amazon EC2, follow the steps in Configure Custom Log Collection to set service.name.
• If your applications are running on Amazon ECS-on-AWS Fargate, follow the steps in Amazon ECS-on-AWS Fargate Application Logs.
• If your applications are running on Amazon ECS-on-EC2, follow the steps in Amazon ECS-on-EC2 Application Logs.

Logs from Applications Running in Non-Containerized Environments

Cisco Cloud Observability only supports log collection from applications running on Amazon EC2. To correlate these logs with their respective machine entities, follow the steps in Amazon EC2 Application Logs to set  service.name. The other attribute that is required for correlation  is service.instance.id which the Log Collector service automatically sets to the EC2 instance ID.

Logs from Functions-as-a-Service (FaaS)

Cisco Cloud Observability only supports log collection from AWS Lambda. To correlate these logs with their respective cloud resources, follow the steps in AWS Lambda Service Logs.

Logs Ingested Using OpenTelemetry

You can correlate OTLP-formatted log messages from an OpenTelemetry™ -instrumented application to Kubernetes Cluster, Namespace, Workload, and Pod entities by embedding the container.id attribute in each message's OTLP resource packet:

• If your application is instrumented with a language-specific OpenTelemetry SDK, see Infrastructure Correlation.
• If your application is a non-Java application or a Java application using manual instrumentation, you can add  container.id through an environment variable, which avoids the need to add SDK calls to a service. See Enable Infrastructure Correlation for Non-Java OpenTelemetry Instrumented Services.

OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.