Kafka brokers are servers that enable users to produce and consume messages by topic.

You must add Kafka exporters to your deployment to view this entity. See Configure Kafka.

Cisco Cloud Observability displays Apache Kafka entities on the Observe page. Metrics display for specific entity instances in the list and detail views.

Detail View 

To display the detail view for a Kafka broker:

  1. Navigate to the Observe page.
  2. Under Infrastructure, click Kafka Cluster.
    The Kafka Clusters list view now displays.
  3. In the Relationships map in the left panel, click Brokers.
    The Brokers list view now displays. The Relationships panel can include non-Kafka entities, such as entities from AWS, Azure, and Kubernetes.
  4. Click a Broker Name to display the detail view for the selected instance.
    The detail view displays metrics, key performance indicators, and properties (attributes) related to the instance you selected.

Metrics and Key Performance Indicators 

Cisco Cloud Observability displays the following metrics and key performance indicators (KPIs) for Kafka brokers.

Display NameDescription
ISR Expands/Shrinks (Count/Sec)The rate per second at which the pool of in-sync replicas (ISRs) expands and shrinks.
Produce Purgatory Size (Count)The purge interval (in number of requests) of the producer request purgatory.
Fetch Purgatory Size (Count)The purge interval (in number of requests) of the fetch request purgatory.
Messages In per SecThe incoming message rate per second.
Bytes In per Sec (MiB/Sec)The inbound rate per second by topic and broker in MiB.
Bytes Out per Sec (MiB/Sec)The outbound rate per second by topic and broker in MiB.
Total Produce Requests per Sec By TopicThe producer request rate per second by topic.
Total Fetch Requests per Sec By TopicThe fetch request rate per second by topic.
Failed Request per SecThe failed request rate per second by broker.
Total Time (ms)

The total time in ms to serve the specified request.

Network Requests per Sec By TypeThe total network requests made for the request per second (for example, Produce, FetchConsumers, and FetchFollower).
Log Size (GiB)The disk consumption in GiB by the topic logs on the broker.
Threads (Count)The current number of threads in use compared to the number of daemon threads.
File Descriptors (Count)The number of open file descriptors compared to the maximum number of file descriptors.
Heap Memory (GiB)The heap memory used vs. heap memory usage committed.
ZooKeeper Metrics (Count/Sec)A multi-line metric that consists of sync connections per second, expires per second, read-only connects per second, and disconnects per second.

Properties (Attributes)

Cisco Cloud Observability displays the following properties for Kafka brokers.

Display Name

Description

Broker Name

The name of the Kubernetes pod.

Broker Host/IP

The broker IP.

Kubernetes Pod UID

The Kubernetes pod UID.

Kafka Cluster Name

The Kafka cluster name.

Kubernetes Cluster Name

The Kubernetes cluster name.

Kubernetes Namespace

The Kubernetes namespace name.

Retention and Purge Time-To-Live (TTL)

For all cloud and infrastructure entities, the retention TTL is 180 minutes (3 hours) and the purge TTL is 525,600 minutes (365 days). 

KAFKA is a registered trademark of The Apache Software Foundation and has been licensed for use by Splunk AppDynamics and its affiliates (together, "Cisco AppDynamics"). Splunk AppDynamics has no affiliation with and is not endorsed by The Apache Software Foundation.

Prometheus® and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.