Cisco Cloud Observability provides full-stack observability for large, managed Kubernetes deployments on public clouds, with support for account and user onboarding, cloud onboarding, infrastructure onboarding, and installation of OpenTelemetry-compatible agents to instrument your microservices. These release notes describe enhancements, new features, resolved issues, and known issues in every release of this product.

As of November 27, 2023, the Cisco Full-Stack Observability (FSO) Platform is now the Cisco Observability Platform, and Cloud Native Application Observability is now Cisco Cloud Observability powered by the Cisco Observability Platform. These name changes better align our products with the Cisco portfolio and with our business strategy.

Release Versions and Dates

In the following tables, the Version column contains the release ID. Release IDs are comprised of the release date and a build number. For example, 22.2.18-1925 indicates a release date of February 18, 2022, and a build number of 1925.

Supported Browser Versions

For the best experience, use the latest version of your browser. Always apply any update available to your browser. Cisco Cloud Observability works best with Chrome, Safari, and the latest version of Microsoft Edge. It is incompatible with older versions of Microsoft Edge.

Third-party Documentation

This document contains references to third-party documentation. Cisco AppDynamics does not own any rights and assumes no responsibility for the accuracy or completeness of such third-party documentation.

April Release - 24.4.x 

Collectors, Orchestrators, Agents, and Add-ons

ArtifactSummary
Kubernetes and App Service Monitoring EKS Addon

There is no 24.4.0 release yet.

Cluster CollectorThere is no 24.4.0 release yet.
Infrastructure CollectorThere is no 24.4.0 release yet.
Log CollectorThere is no 24.4.0 release yet.

Cisco AppDynamics Distribution of OpenTelemetry Collector

There is no 24.4.0 release yet.

AppDynamics Operator

There is no 24.4.0 release yet.

OpenTelemetry Operator for Kubernetes

There is no 24.4.0 release yet.

Cisco AppDynamics Smart Agent

There is no 24.4.0 release yet.


Coming Soon in 24.4

The software features described below are projected but not guaranteed to be offered in the next release, and therefore should not be interpreted as a promise of future functionality. Cisco AppDynamics makes no warranties regarding these features, including performance, quality, or availability, and reserves the right to change the features at any time before making them generally available as well as never making them generally available. Any buying decisions should be made based on features and products that are currently generally available.

UI Enhancements

CategorySummaryVersion
App Root Cause Analysis using Anomaly Detection

Cisco Cloud Observability can now identify the underlying root causes of an anomaly by correlating infrastructure issues with application issues. For example, if a business transaction anomaly is detected and its root cause is a host or pod issue, Cisco Cloud Observability automatically pinpoint the suspected cause within the host or pod.

24.4.0
Alerting

In past versions, Cisco Cloud Observability had many predefined health rules configured with various MELT data types, conditions, and thresholds against the baseline. In this release, to reduce unnecessary alerts triggered by health rules, only the critical health rules and the health rules that are already modified or saved without any modification by users are enabled by default. Any redundant or duplicate predefined health rules are deprecated.

24.4.0

You can now test an HTTP action to a configured endpoint and check if the action is working instead of waiting for an actual violation. After testing, you get a success or failure message along with the status code that helps you to debug any issue that might occur while testing the HTTP action.

24.4.0

On the entity centric page, you can now view all the alerts corresponding to an entity by selecting a specific time range or the entire time range on the Entity Health Timeline graph. Click each alert to view its details and the stages of the alert such as violation started, violation upgraded, violation downgraded, violation continues, and so on.

24.4.0

In an HTTP action payload, you can now fetch the values of the attributes of an entity. To fetch the values in runtime, use ${entityAttrList[0].<attribute_name>} in the payload.

24.4.0

On the Observe page and the entity centric pages, the health icon NO ALERT RULES ENABLED is displayed for the entities that don’t have any health rules configured or when the corresponding health rules are deleted.

24.4.0
Cloud and Infrastructure Monitoring

This release supports a new AWS Events Ingestion feature. You can now send events from Amazon EventBridge to Cisco Cloud Observability and monitor AWS events using the Explore > Events page.

24.4.0

Cisco Cloud Observability now supports monitoring Amazon DocumentDB.

24.4.0

Cisco Cloud Observability now supports monitoring Azure Event Hubs.

24.4.0

Cisco Cloud Observability now supports monitoring:

  • GCP Cloud Tasks
  • GCP Compute Engine Autoscaler
  • GCP Identity and Access Management (IAM)
  • GCP Memorystore for Memcached
24.4.0
Dashboards

Cisco AppDynamics now offers Azure and Service Performance Overview dashboards to possibly detect problems before they happen, while viewing multiple entity types and manipulating data to fit your needs.

24.4.0
Kubernetes and App Service Monitoring Collectors

This release introduces the new global parameter clientIdEnvVar that enables you to use the secret key reference for the client ID.

For example:

oauth:
    #clientId: agt_2qCuuD07AXwzPZYnhmZs
    clientIdEnvVar:
      valueFrom:
        secretKeyRef:
          name: "custom-secret-2"
          key: "custom-controller-key-2"
CODE
24.4.0
Logs

For log ingestion from Amazon EC2 Application Logs, you can now specify the EC2 name in AGENT_PLATFORM_NAME

24.4.0
Network Monitoring

This release adds the following features for Network Monitoring:

  • You can now monitor network connections on the list and detail view of the Namespaces entity-centric page.
  • On the Clusters detail view, you can use the new Scope drop-down menu to switch the scope of network monitoring to namespace or workload. The flow map and connection list will be updated based on your selection.
24.4.0
Overview Page

The Overview page has a new Get Started panel that asks you questions to assist you better with onboarding Cisco Cloud Observability.  There is also a new Recently Visited that displays the previous 8 pages you visited, and a Restore Page button to restore Overview to its default settings. See Overview.

24.4.0
Platform Monitoring

The Platform Monitoring dashboard now supports filtering your Tenant's telemetry so that you see only what applies to a specific solution or entity.

24.4.0
Schema Browser

You can now view the number of active instances of an entity. On the Explore area, click an entity to view the Number of instances.

24.4.0
Service-oriented InvestigationsThere are new trace and span usability and filter enhancements. 24.4.0
There are new enhancements for JMX metrics.24.4.0
Traces

You can now configure backend trace sampling to reduce your storage costs. Configuration is available only through the Knowledge Store API

24.4.0

Resolved Issues

KeySummaryVersion
ALERT-15541

Add health monitoring support for Kubernetes-related entities that are associated through relationships.

24.4.0
PRE-18086Anomaly Detection is not working as expected: Existing data is displayed as unavailable. 24.4.0
PRE-16626

The alerting services are receiving notifications of deleted events for entities too late.

24.4.0

Known Issues and Limitations

CategorySummary

Customer Onboarding and Startup

Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

Logs

Logs for Amazon ECS entities don't appear on the entity-centric pages (ECPs). Workaround: Navigate to Explore > Logs to search for these logs. 

Network Monitoring

For larger deployments, network connection flow maps may not display all of the arrows that represent the incoming or outgoing network connections for a workload. Workaround: Use the List tab to view all network connections.

Service-oriented Investigations

Currently, natural language filtering while using a Favorites shortcut on the Business Transactions (BT) list page is not supported. For example, if you select a BT favorited shortcut and then apply a natural language filter, it removes the favorited filter and only keeps the natural language filter.

Module Enhancements

Module NameSummaryVersion

Cloud Cost Management





The new Cloud Cost Management module replaces the Cost Insights and Application modules but includes and unites their functionality and features. 

24.4.0

The  24.4.0 release improves the application resource optimizer's handling of platform internal failures and allows it to recover from temporary outages in platform services. In some cases, the optimization process may restart (whereas previously, it may have left the optimization in a state that requires manual intervention).

In addition, the Cloud Management expands Cost Insights for the following services:

OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.

March Release - 24.3

Enhancements

CategorySummaryVersion

Agent Management

In the 24.2.0 release, the Infrastructure Collector was displayed on the Agent Management console as two collectors: Server Collector and Container Collector. In this release, the Infrastructure Collector is now displayed as a single collector under the name Infra Collector.

This is a breaking change. Action is required.

In the Metric Explorer, historical health metrics for the Server and Container Collectors will no longer be accessible. To view the Infrastructure Collector in the Agent Management console and to view its metrics in the Metric Explorer, you must upgrade your operators and collectors to version >=24.3.0.

24.3.0

Alerting




For a data-unavailable scenario, you can now control the health rule evaluation to one of the following:

  • Critical or Warning: The health rule considers this no data scenario as Critical or Warning condition and the health rule status is shown in red or yellow respectively.
  • Unknown: The health rule considers the no data scenario as unknown and the health status is shown in gray.
  • Healthy: The health rule considers the no data scenario as healthy and the heath rule status is shown in green.
24.3.0

From the list view, you can now view the health violation information of the entities that are in critical, warning, or unknown state. Click the health icon to view the violating metrics and the conditions on the HEALTH AND ALERTING inspector panel. For example, if an entity is in a critical state, you can click the health icon to view violating health rules or anomaly detection. This information helps you quickly identify the entity's violations without navigating deeper into the entity page.

24.3.0

When you link an action to a health rule of an entity, the action applies to only the leaf entity and not to the parent entities in an entity scope path. For example, when you create a health rule for an entity type k8s:pod and select Entity scope as k8s:pd->k8s:cronjob→k8s:namespace, the action that you select for this health rule applies to the leaf entity k8s:pod.

24.3.0

App Root Cause Analysis using Anomaly Detection

While configuring Anomaly Detection for an entity, you can now select the option to evaluate an entity's health for a no data scenario. Select the health evaluation as Unknown or Healthy when no metric data is available for evaluation.

24.3.0

Collectors and Operators

This release includes the following artifacts:

  • OTel Docker image: 24.3.0-1525
  • Log Collector Docker image: 24.3.0-1066
  • Cluster Collector: 24.3.0-1968
  • Infrastructure Collectors Docker image for Kubernetes and cloud infrastructure: 24.3.0-1968
  • Infrastructure Collectors Docker image for ECS: 24.2.0-5118
  • Cisco AppDynamics Operator Docker image: 24.3.0-1339
  • Cisco AppDynamics Operator Hub bundle: 24.3.0-111
  • Cisco AppDynamics Collectors Helm chart: 1.20.1117
  • Cisco AppDynamics Operator Helm chart: 1.20.299
  • Cisco AppDynamics Operator for Spectro Cloud Helm chart: 1.20.299
  • Cisco AppDynamics Collector for Spectro Cloud Helm chart: 1.20.1117
24.3.0

Cisco AppDynamics Support for OpenTelemetry



This release adds the following new configurations for the Cisco AppDynamics Distribution of OpenTelemetry Collector:

24.3.0

In the configOverride configuration, the data type of retry_on_failure.max_elapsed_time was changed to time.Duration. If you manually configured retry_on_failure.max_elapsed_time as an integer, you must convert it to time.Duration.

Example

exporters: 
  otlphttp: 
    retry_on_failure: 
# max_elapsed_time: 180 should be changed to
       max_elapsed_time: 180s 
YML
24.3.0

This release adds a new batchbybytesize processor. This processor breaks large OTLP data packets into smaller packets and prevents Cisco Cloud Observability from rejecting data packets due to size limitations. See Configure Max Export Byte Size.


Cloud and Infrastructure Monitoring

Cisco Cloud Observability now supports monitoring Azure Container Apps.

24.3.0
24.3.0

Documentation

This release includes a new quick start guide for Kubernetes and App Service Monitoring. This installation flow is supported for Amazon Elastic Kubernetes Service, Azure Kubernetes Service, Google Kubernetes Engine, and Red Hat OpenShift.

24.3.0

Logs

Documentation updates:

24.3.0

Metrics

The Cisco Observability Platform now handles inexact intervals between metrics. It aggregates metrics collected at sub-minute granularities into one-minute granularities by default.

24.3.0
Platform Monitoring

In 24.1, Cisco AppDynamics subscribed all Tenants to the Platform Monitoring dashboard. This dashboard provides visibility into the runtime status of the Cisco Observability Platform. In doing so, it consumes your metric tokens to collect telemetry related to your Tenant's runtime environment. Therefore, Cisco AppDynamics has now unsubscribed all Tenants from this dashboard and made it an opt-in choice instead. To opt in, ask Customer Support to resubscribe your Tenant. Resubscribing to this dashboard will again incur the usage of your metric tokens.

24.3.0
Service-oriented Investigations

Cisco AppDynamics now supports .NET for for the downstream Lambda entry span. See 

Monitor AWS Lambda Using OpenTelemetry Instrumentation.

24.3.0

The performance of tail sampling collector metrics has been updated. See OpenTelemetry Trace Sampling.

24.3.0

The system CPU utilization for Java Agent 2.0 users has been updated. See Service Instances Detail.

24.3.0

You can now see Span and Traces list and detail view in the BRUM application, as well as in the Cisco Observability Platform application. See Traces View Details.

24.3.0

For the spans and traces pages, there were many bug fixes including updated Duration numbers, time range enhancements, and recent filters fixes. There is also a new “Examples for Filter” tip that is directly on the UI next to the Spans or Traces name at the top of the Spans or Traces page.

24.3.0

You can now easily sort metrics on all APM pages.

24.3.0

There is now a new Service Endpoints list and detail page where you can view greater details on the Service Endpoints. See Services Detail.

24.3.0

You can now see end-to-end Errors/min for the entire Business Transaction, from the root span to all of its children, which displays a more holistic metric.

24.3.0

The Observe page has been updated with a new design. See Observe UI Overview.

24.3.0

Resolved Issues

KeySummaryVersion

PRE-16782, ALERT-15455

In alerting, if Cisco Cloud Observability receives the same metric for a specific entity more than once within a minute, Cisco Cloud Observability identifies it as a duplicate. This issue has been resolved.

24.3.0

PRE-17077, DASHUI-7863

When clicking on a Service Endpoint from the Health Rule Configuration List, you were taken to a screen that states "The endpoint view is not supported!" This issue is now resolved.24.3.0

PRE-16271

There were no metrics in Prod cell 4. Metrics are now present. 

24.3.0

PRE-16844, DASHUI-7377

Span attributes were not being displayed in the list in the filter bar. This issue is now resolved.24.3.0

PRE-17048, DASHUI-6617

If Business Transactions names were long, the names extended beyond the time range selector. This issue has been fixed.24.3.0

Known Issues and Limitations

CategorySummary

Customer Onboarding and Startup

Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

Logs

Logs for Amazon ECS entities don't appear on the entity-centric pages (ECPs). Workaround: Navigate to Explore > Logs to search for these logs. 

Network Monitoring

For larger deployments, network connection flow maps may not display all of the arrows that represent the incoming or outgoing network connections for a workload. Workaround: Use the List tab to view all network connections.

Service-oriented Investigations

Currently, natural language filtering while using a Favorites shortcut on the Business Transactions (BT) list page is not supported. For example, if you select a BT favorited shortcut and then apply a natural language filter, it removes the favorited filter and only keeps the natural language filter.

Module Enhancements

CategorySummaryVersion

Extend Cisco Cloud Observability Modules 

Cost Insights has the following enhancements for the 24.3 release:

24.3.0

Extend Cisco Cloud Observability Modules 

In the 24.3 release, the Application Resource Optimizer has the following new features:

Instant Recommendations 

Instant Recommendations are suggestions for right-sizing the resource specification of Kubernetes deployments based on recent actual resource use. See Instant Recommendations.

Optimize Tab In The Kubernetes Workloads List

The Application Resource Optimizer has the new tab Optimize, which shows a summary of performance and cost optimization fields, including efficiency rate, reliability risk, available savings, and eligibility for active optimization.

See Choose Your Workload to learn how to use the Optimize tab to select a workload to optimize. 

Support for Optimizing Graviton Workloads

The ARO Servo Agent can now run on Graviton-only clusters to optimize workloads for this ARM64 architecture. 

24.3.0

OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.

February Release - 24.2.x 

Enhancements

CategorySummaryVersion

Cloud and Infrastructure Monitoring

Cisco Cloud Observability now supports monitoring Amazon SageMaker.

24.2.29

Cisco Cloud Observability now supports monitoring:

24.2.29


Collectors and Operators



This release includes the following artifacts:

  • OTel Docker image: 24.2.0-1451
  • Log Collector Docker image: 24.2.0-1050
  • Cluster Collector: 24.2.0-1923
  • Infrastructure Collectors Docker image for Kubernetes and cloud infrastructure: 24.2.0-1923
  • Infrastructure Collectors Docker image for ECS: 24.2.0-5093
  • Cisco AppDynamics Operator Docker image: 24.2.0-1303
  • Database Collector Docker image: 24.2.0-1084
  • Cisco AppDynamics Collectors Helm chart: 1.19.1062
  • Cisco AppDynamics Operator Helm chart: 1.19.289
  • Cisco AppDynamics Operator for Spectro Cloud Helm chart: 1.19.289
  • Cisco AppDynamics Collector for Spectro Cloud Helm chart: 1.19.1062
  • Cisco AppDynamics Smart Agent: 24.2.0-1529
24.2.29

The Cisco Cloud Observability Helm charts are tested as compatible with Helm v3.13.x. 

24.2.29

This release supports Kubernetes nodes running on ARM64 architecture for these components:

  • Cisco AppDynamics Smart Agent
  • Cisco AppDynamics Operator
  • Cluster Collector
  • Infrastructure Collector
  • Cisco AppDynamics Distribution of OpenTelemetry Collector
  • Log Collector

The OpenTelemetry Operator for Kubernetes supports ARM64.

24.2.29

Database Monitoring

This release adds:

24.2.29

Extend Cisco Cloud Observability Modules 

You can now integrate Splunk Enterprise with Cisco AIOps. This integration brings the Splunk alerts into Cisco AIOps to correlate it with data from other monitoring tools. See 2024-04-22_23-36-05_Splunk Integration.

24.2.29

Log Monitoring

You can now forward your Amazon S3 access logs to Cisco Cloud Observability. See Amazon S3 Access Logs.

24.2.29

The Log Collector has renamed one of the Kubernetes®  properties which it automatically extracts based on the value of the clusterName parameter in collectors-values.yaml: k8s.container.name is now container.name. This change doesn't affect your collectors-values.yaml.

24.2.29

New Support for Non-Kubernetes Log Collector Observability:
You can now send  metrics from the Log Collector service running on an Amazon EC2 instance to Cisco Cloud Observability. This Log Collector service is not visible on the Agent Management console because this console currently only displays Kubernetes-based collectors, but you can find the metrics associated with this Log Collector service using UQL queries. See Amazon EC2 Application Logs.

24.2.29
All queries to the backend log store now fetch a maximum of 1000 most-recent fields per index to prevent giant queries from slowing down the UI. See Explore Logs.
24.2.29

Documentation updates:

24.2.29

Metrics

The new Explore > Metric Explorer page provides a comprehensive tool for exploring metrics from any domain or entity. Previously, you could only view a limited subset of the metrics that were associated with an entity on its detail view; this view didn't display dynamic metrics, metrics that are associated with other entities, or even all of the metrics that are associated with that entity. Now, you can use the Metric Explorer to see the bigger picture of your environment. See Explore Metrics.

24.2.29

Network Monitoring

This release supports a new Network Monitoring feature for Kubernetes clusters and workloads. This feature uses eBPF to:

  • Visualize and monitor the incoming and outgoing TCP/UDP network connections in your environment.
  • Display the metrics and key performance indicators, properties, and health status of each network connection.
  • Enable you to identify performance bottlenecks and their impact on the performance of your applications.
24.2.29

Service-oriented Investigations

For services, there are now new metric cards that are sorted by Average Response Time (ART), Calls Per Minute (CPM), and Errors Per Minute (EPM). See Services. There are also new flow map enhancements including a new feature where particles flow through neighboring nodes and now when you apply a filter, the flow map now zooms in on the filtered nodes. See Services Detail.

24.2.29


Resolved Issues

KeySummaryVersion

ALERT-15539

The health rule keeps getting violated even though the metric has not reached the threshold.24.2.29

CLOUDMON-3729

The creation of cloud connections via cloudformation template was failing. This issue is now resolved.24.2.29

DASHUI-7016

The span attributes are not populated in the span filter list screen24.2.29

O2-2884

The POD status was not reflecting the correct response. This issue is now resolved.  24.2.29

PRE-13852

There were issues with Aurora-mysql context and page loading across database monitoring screens. This issue is now resolved. 24.2.29

PRE-15548

The Log Collector's default and custom configurations on Amazon EC2 was not being enabled or disabled correctly. This is now resolved. 24.2.29

PRE-16696

The Cisco AppDynamics Operator pods kept crashing when any parser other than timestamp was enabled inside the defaultConfig section of logCollectorConfig. This is now resolved.24.2.29

TP-878

TP was processing traces that only had at least one root span (span with no parent). There were instances where TP received a trace with no root span. In such cases, TP now continues to process the partial trace and and create topology and measurement data. This issue is now resolved.24.2.29

XTNSBL-3324

For the Database Cluster UI, the Name column was displaying the Resource ID. This issue is now resolved. 24.2.29

XTNSBL-3821

UQL was missing relationship between the Aurora cluster and Aurora instance. The relationship has now been fixed and this issue is resolved. 24.2.29

XTNSBL-4259

Some metrics were missing for the Azure App Service. This issue is now resolved. 24.2.29

Known Issues and Limitations

CategorySummary

Customer Onboarding and Startup

Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

Kubernetes and App Service Monitoring

The Kubernetes and App Server Monitoring installation fails when you use the 1.18.983 version of Cisco AppDynamics Collector for Spectro Cloud Helm chart. 

To workaround this issue, include the oauth parameter under global section in the values file when adding the collector pack:

global:
  clusterName: <cluster-name>
  oauth:
    clientId: <client-ID>
    clientSecret: <client-secret>
    endpoint: <endpoint-URL>
    tokenUrl: <token-URL>

CODE

For information about adding the collector pack, see Install Kubernetes and App Service Monitoring with Spectro Cloud Palette.

Logs

Logs for Amazon ECS service entities don't appear on the entity-centric pages (ECPs). Workaround: Navigate to Explore > Logs to search for these logs. 

Network Monitoring

For larger deployments, network connection flow maps may not display all of the arrows that represent the incoming or outgoing network connections for a workload. Workaround: Use the List tab to view all network connections.

Service-oriented Investigations

Currently, natural language filtering while using a Favorites shortcut on the Business Transactions (BT) list page is not supported. For example, if you select a BT favorited shortcut and then apply a natural language filter, it removes the favorited filter and only keeps the natural language filter.

Azure Load Balancers and Application Gateway

When navigating to Observe > Load Balancers > Azure or Observe > Load Balancers > Application Gateway, you cannot view the details of the Azure Load Balancer and Application Gateway entities in the Property panel. 

Module Enhancements

CategorySummaryVersion

Extend Cisco Cloud Observability Modules 

Cost Insights has the following enhancements for the 24.2 release:

  • Expanded support for AWS and Azure services for billing integration:
    • Elastic Load Balancing (ELB)
    • Amazon Relational Database Service (RDS)

    • Azure Load Balancer
    • Azure Application Gateway
  • Support for billing integration with Azure.
  • Improved display of the billing credentials in the Settings > List view.
24.2.29

Extend Cisco Cloud Observability Modules 

You can now subscribe to the new Service Level Management (SLM) module. SLM allows you to:

  • Track reliability of one or more Business Transactions (BTs) by defining Service Level Objectives (SLOs). 
  • Supports SLOs on BT's latency and availability indicators.
  • Visualize trends and allows defining alerts based on error budget consumed or burn rates.

See Service Level Management.

24.2.29




OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.

January Release - 24.1.x 

Enhancements

CategorySummaryVersion
Agent Management

Cisco Cloud Observability has a new Agent Management console that can be used to view the active Cisco AppDynamics agents and collectors in your environment. 

24.1.31
Alerting

When you create a health rule, you can now use the like (~) operator to filter an entity type by tags, metric attribute, or both that matches a specific pattern. The like operator searches for a specified  pattern in the available data. You can use the like operator with the wildcard character asterisk (*). The search is case-sensitive.

For examples:

service.name ~ '*payment'

Searches all the values of the attribute service.name that end with payment such as, service1-payment and service2-payment.

service.name ~ 'payment*'

Searches all the values of the attribute service.name that begin with payment such as, payment-service-1, payment-service-2.

service.name ~ '*payment*’

Searches values of the attribute service.name that contain payment such as, service1-payment , payment-service-2, and orderpaymentservice.

See Define Entity Details and Define Health Rule Evaluation Conditions.

24.1.31
Application Performance Monitoring with OpenTelemetry™

Cisco Cloud Observability now offers a trace sampling solution that enables you to use head sampling (at this time, for Java only), tail sampling, or both. For both of these sampling options, Cisco Cloud Observability uses probabilistic sampling. See OpenTelemetry Trace Sampling.

24.1.31
Cisco Cloud Observability Kubernetes Collectors

This release introduces new global parameters that is required to install the collectors for Agent Management. If you have deployed Cisco Cloud Observability Kubernetes Collectors (Cluster, Infrastructure, and Database) on your Kubernetes clusters, you must add the following new parameters to the global section of your collectors-values.yaml, replacing <tenant-id> with the your tenant ID, which is displayed on Configure > Kubernetes and APM. See Configure Cisco AppDynamics Kubernetes Collectors.

oauth:
   clientId:
   clientSecret:
   endpoint: https://<your-tenant-url>/data
   tokenUrl: https://<your-tenant-url>/auth/<tenant-id>/default/oauth2/token  
YML
24.1.31
Cloud Infrastructure and Troubleshooting

Cisco Cloud Observability now supports monitoring Google Cloud Platform™ (GCP) BigQuery.

24.1.31

Cisco Cloud Observability now supports monitoring the following Amazon Web Services (AWS) services:

24.1.31
Collectors and Operators

The Cluster Collector 24.1.1-1915 fixes an intermittent issue where data collection may not happen every minute for some clusters.


24.2.15

The following helm charts fix the issues with using the clientSecretEnvVar parameter under the global.oauth collectors configuration:

  • AppDynamics Collectors Helm chart: 1.18.983
  • AppDynamics Collector for Spectro Cloud Helm chart: 1.18.983
24.2.2

This release includes the following AppDynamics artifacts: 

  • OTel Docker image:
    • Linux: 24.1.0-1297
    • Windows: 24.1.0-1297-windows-amd64-nanoserver-ltsc2019
  • Log Collector Docker image: 24.1.0-970
  • Cluster Collector: 24.1.0-1887
  • Infrastructure Collectors Docker image for Kubernetes and cloud infrastructure: 24.1.0-1887
  • AppDynamics Operator Docker image: 24.1.0-1230
  • OperatorHub bundle: 24.1.0-90
  • Database Collector Docker image: 24.1.0-1046
  • AppDynamics Collectors Helm chart: 1.18.972
  • AppDynamics Operator Helm chart: 1.18.265
  • AppDynamics Operator for Spectro Cloud Helm chart: 1.18.265
  • AppDynamics Collector for Spectro Cloud Helm chart: 1.18.972
  • AppDynamics Smart Agent: 24.1.0-1431
24.1.31
Dashboards

Cisco Cloud Observability now provides visibility into your cloud operations via dashboards. Powered by Infrastructure Collector and CloudWatch, dashboards are instrumental in monitoring Cloud Services. You can view the services you are using as well as aggregate data, all in one place. See Dashboards.

24.1.31
Database Monitoring

Simplified the Cisco AppDynamics Operator installation process while deploying Database Monitoring. The installation process is the same as Install Kubernetes and App Service Monitoring.

24.1.31
EventsThe Explore > Events page is new in this release. It provides a unified events explorer that brings together events from multiple domains, providing a holistic view of the system's behavior and performance and allowing you to understand the relationships and dependencies between events across different components, enabling comprehensive visibility and efficient troubleshooting.24.1.31
Extend Cisco Cloud Observability Modules 

Cisco AIOps is a cloud solution that provides an end-to-end visualization of the applications and infrastructures, incident correlation, event monitoring, and actions. You get a unified view of all operational domains in a single pane. The actionable and correlated insights help to reduce the mean time to detect (MTTD) and mean time to resolve (MTTR) issues.

Access Cisco AIOps as a Cisco Cloud Observability module, which is built on Cisco Observability Platform. See Cisco AIOps.

24.1.31
General

Cisco Cloud Observability has a new Overview page where you can explore a Welcome Tour, recommended setup steps, product features, the Cisco Observability Platform Exchange, and help resources. There is also a new application switcher where you can navigate between apps. See Overview.

24.1.31
LogsThe Configure > Log Parsing and Configure > Log Processing pages now provide predefined GROK parsing patterns for Apache, Amazon ALB, Amazon ELB, AWS S3 Access, HA Proxy, Kafka, MySQL, NGINX, PostresSql, Redis, SQL Server, and Zookeeper. Predefined patterns reduce complexity with extracting the fields you need from your log messages and speed up troubleshooting. You can use these patterns in your pre-ingestion and post-ingestion (server-side) parsing configurations. See Log Parsing Validator and Log Processing Rules.24.1.31

The Logs page now has a natural language processing (NLP) algorithm that recognizes queries written in a style more similar to how we talk as humans. NLP-powered searching removes the need for you to translate what you're really looking for into a query syntax. Removing this extra translation step speeds up troubleshooting.

24.1.31

If you've deployed the Log Collector on an Amazon ECS cluster with Amazon EC2 infrastructure, you must update your task definition to correct a bug. See Amazon ECS-on-EC2 Application Logs.

24.1.31
Platform Monitoring

Platform Home > Tools > Platform Monitoring is a dashboard that provides visibility into the runtime status of the platform. Tenants can use this dashboard to drill down into the platform's telemetry related to the three segments of MELT data pipeline:

  • Data ingestion
  • Data mapping and processing
  • Data storage Previously, when tenants sent data to the CIS endpoint they wouldn't know what happened to their data after it reached the CIS endpoint. With this dashboard, Tenants have insight into any errors that happen beyond the CIS endpoint.
24.1.31

Service-oriented Investigations




There are now linked spans icons and a Linked Spans dropdown menu in the Properties panel for entities related to Lambda. Also, other then APM entities, there are more entity types that can generate spans including BRUM entities. See Traces View Details.

24.1.31

There are new flow map enhancements including a click-and-drag feature to separate nodes from each other and new flow map options on a node or interaction's detail page including Rebalance Nodes, Smooth Edges, Truncate Labels. There are also new navigation controls. See Service Instances Detail, Services Detail, and Business Transactions Detail.

24.1.31

You can now use Natural Language to filter your entities. Natural Language filtering involves entering a filter statement using human language. For this release, you cannot use time ranges with Natural Language filtering, for example, "Show unknown services in the last 2 hours". Using Natural Language with Group view (Classic filter feature) or entering "sort by" as a part of the filter statement are not supported. See Filters.

24.1.31

Resolved Issues

KeySummaryVersion
DASHUI-7377Span attributes were not being displayed in the spans filter.24.1.31


Known Issues and Limitations

CategorySummary

Customer Onboarding and Startup

Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

Logs

Logs for Amazon ECS entities don't appear on the entity-centric pages (ECPs). Workaround: Navigate to Explore > Logs to search for these logs. 

Service-oriented Investigations

Natural language filtering while using a Favorites shortcut on the Business Transactions (BT) list page is not supported. For example, if you select a BT favorited shortcut and then apply a natural language filter, it removes the favorited filter and only keeps the natural language filter.

Module Enhancements

CategorySummaryVersion

Extend Cisco Cloud Observability Modules 

The Kubernetes Cost and Workload Profiler now displays its analysis on the Profiler tab of the Application Resource Optimizer widget instead of the Efficiency Profile widget. See Efficiency and Risk Profile View.

24.1.31

Extend Cisco Cloud Observability Modules 

Application Resource Optimizer (ARO) has the following updates for 24.1.0:

  • ARO has been integrated with Cost Insights. If you also have subscribed to Cost Insights and have an active license, the monthly savings for a deployment will be displayed in USD and also as a percentage.

    The projected cost savings are based on the monthly cost of the deployment at the same average number of replicas as the deployment had in the previous seven days. Cost Insights, not ARO, determines the projected costs.

  • You no longer use the Optimize button from the Efficiency Profile widget to start optimizing your Kubernetes workloads. Instead, you navigate to the Application Resource Optimizer widget and select the Active Optimization tab to begin the optimization process.

24.1.31

Extend Cisco Cloud Observability  Modules 

Cost Insights has added the following features for the 24.1.0 release:

  • You can integrate your cloud provider's bill data to calculate your Kubernetes costs accurately. By default, Cost Insights bases costs on the cloud provider's publicly available price listing to calculate your Kubernetes costs. 
  • You can view cost information for business transactions and these Amazon services through their respective entities and the Property Panel:
    • Elastic Load Balancing (ELB)
    • AWS Lambda
    • Amazon Simple Queue Service (Amazon SQS)

    • Amazon S3

    • Amazon Relational Database Service (RDS)

    • Amazon Elastic File System (EFS)

  • You can view budget information in the Teams entity, which simplifies the UI, more tightly integrates the budget teams, and creates a one-to-one relationship between a team and a budget.  
24.1.31

Extend Cisco Cloud Observability Modules 

Cisco Secure Application now allows you to view Cisco Secure Application collectors in the Agent Management console. See Agent Management.

24.1.31




OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.

December Release - 23.12.x 

Enhancements

CategorySummaryVersion

General

All system components and collectors, and all communication into, out of, and within the platform, now use TLS protocol version 1.3 by default.

23.12.14

Alerting

You can now create health rule conditions for logs and get alerts for abnormal logs. You can also apply filters based on the log attributes. On the entity-centric page, you can view the Violating Events graph and Violation Details related to the log. 

The Violating Logs chart provides a graphical view of the following information:

  • The total number of logs that occurred during the look-back period. The look-back period is specified in the  Use data from last <> min(s) option while configuring a health rule. The health rule continuously evaluates on a rolling period, fetches the count of logs in this rolling period, and alerts if the log count deviates from the threshold.
  • The trend of the log count.
  • The threshold value and the log counts that are deviating from the threshold.

See Health Violation Timeline.

23.12.14

App Root Cause Analysis using Anomaly Detection

On the entity-centric page, the anomaly status is displayed as unknown when data is unavailable for anomaly evaluation. The Anomaly Detection timeline in the Health Violation section, Violating Metrics graph, and  Suspected Cause metrics graph display the no data period as grey.

See Monitor Anomalies.

23.12.14

Cloud Infrastructure and Troubleshooting

Cisco Cloud Observability now supports monitoring the following Google Cloud Platform™ (GCP) services:

23.12.14

Cloud Infrastructure and Troubleshooting

23.12.14, 23.12.21

Collectors and Operators


You can now set priorityClassName in the AppDynamics collector (Kubernetes, Infrastructure, and Database) and operator pods configuration to specify the name of the pod priority class. See Cisco AppDynamics Collectors Settings, Database Collector Settings, and Cisco AppDynamics Operator Settings.

23.12.14

This release includes the following AppDynamics artifacts: 

  • OTel Docker image:
    • Linux: 23.12.0-1247
    • Windows: 23.12.0-1247-windows-amd64-nanoserver-ltsc2019
  • Log Collector Docker image: 23.12.0-874
  • Cluster Collector: 23.12.0-1817
  • Infrastructure Collectors Docker image:
    • ECS: 23.12.0-4907
    • Kubernetes and cloud infrastructure: 23.12.0-1817
  • AppDynamics Operator Docker image: 23.12.0-1173
  • OperatorHub bundle: 23.12.0-45
  • Database Collector Docker image: 23.12.0-1029
  • AppDynamics Collectors Helm chart: 1.17.880
  • AppDynamics Operator Helm chart: 1.17.244
  • AppDynamics Operator for Spectro Cloud Helm chart: 1.17.244
  • AppDynamics Collector for Spectro Cloud Helm chart: 1.17.880
23.12.14

Database Monitoring

This release supports the latest versions of MSSQL and MySQL. See Supported Databases.

23.12.14

Documentation

The following enhancements were made to the Cisco Cloud Observability documentation:

The following new pages were added:

23.12.14

Kubernetes and App Service Monitoring





Cisco Cloud Observability now supports an alternate installation method in Amazon Elastic Kubernetes Service (EKS) clusters. This method is partially automated using Terraform and streamlines the installation process. See Use the Cisco Cloud Observability Amazon EKS Blueprints Add-On.

23.12.14

You can now monitor the ReplicationController workload metrics from the Workloads page. For an OpenShift cluster, you can monitor the underlying ReplicationController created by Deployment Config.

23.12.14

You can install the Cisco AppDynamics Collectors using the AppDynamics OpenShift OperatorHub Bundle. See Install Cisco AppDynamics Collectors with OpenShift OperatorHub Bundle.

23.12.14

Logs




The Log Collector now supports TLS protocol version 1.3 by default. If you want the Log Collector to use a different TLS protocol on following deployments, see the following pages: 

23.12.14

Changes related to deployments on Kubernetes clusters using the advanced format (filebeatYaml) collectors-values.yaml:

  • Improved organization of configuration parameters related to the Filebeat processors.
  • Renamed configuration parameters: All deployments using these parameters must rename them: 
    • output.otlploggrpc.include_resourcesoutput.otlploggrpc.groupby_resource_fields

    • monitoring.otlpmetrc.resource_attrsmonitoring.otlpmetrc.resource_attributes

  • See Log Collector Settings - Advanced YAML Layout.
23.12.14

The Log Collector running on Windows-backed Kubernetes nodes no longer supports x386 architecture.

23.12.14

Service-oriented Investigations



Cisco Cloud Observability now correlates upstream services to downstream Lambda and the SNS or SQS backend types. 

23.12.14

New enhancements include entity icons, hover popups, and opaque transparency percentages for all flow maps (Business Transactions and Services) and Lambda entities.

23.12.14

When a Node.js or Python Lambda interacts with SNS, there is a limitation in the opentelemetry-python-0_2_0 and opentelemetry-nodejs-0_2_0 and older open source layer versions where the open source layer does not send a required field for the backend (the AWS Account ID). In this scenario, Cisco AppDynamics categorizes the SNS AWS Account ID as UNKNOWN.

The work around for the missing Account ID is to manually inject the AWS Account ID. This can be done via the otel collector config file. In the processor section of the config, you can include the attributes of "cloud.account.id". It is important to note that this workaround is applicable only when a unified view of the workflow is desired, and all components of the workflow and setups reside within the same AWS account. The following example demonstrates how to inject the missing attribute in the processors section:

processors:
  attributes:
    actions:
      - key: cloud.account.id
        value: "XXXXXXXXX"
        action: upsert 
CODE
23.12.14


Known Issues and Limitations

CategorySummary

Customer Onboarding and Startup

The service provider-initiated SSO is not working (it does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

Logs 

The backend storage, a time-series data store, drops incoming log messages older than 24 hours because processing old messages in time-bound order would adversely affect its performance.

Resolved Issues

KeySummaryVersion

DSO-725

Cannot enable a connection between the Database Collector and the database instance using a TLS certificate.

23.12.14

LOGDIP-2257

An error is displayed when creating a log parsing rule using the JSON type parser.

23.12.14

TE-278

There was a default value "N/A" in the span view for any span attributes that had a value containing an array of strings because of a code suggesting the UQL response parsing logic was assuming the attributes had a string value. That code has since been updated to properly handle and display attributes with a list value.

23.12.14


Module Enhancements

CategorySummaryVersion
Extend Cisco Cloud Observability Modules 

The Cisco Secure Application release includes: 

  • Business transaction context as part of the business risk factors.
  • A new Detail View for the Vulnerabilities entity-centric page to track affected entities by vulnerability.
  • A new Image entity-centric page to track vulnerabilities affecting each image. 
23.12.14
Extend Cisco Cloud Observability Modules 

The Helm chart deploying the Smart Agent automatically applies labels OPTIMIZER_ID and OPTIMIZED_WORKLOAD_NAME to facilitate tracking and troubleshooting issues with the Application Resource Optimizer. The labels help you identify which objects are created by ARO and can be displayed with most kubectl get commands.

23.12.14
Extend Cisco Cloud Observability Modules 

The Application Resource Optimizer (ARO) release includes:

  • YAML resource configurations: ARO auto-generates YAML snippets with optimized resource configurations that can be copied and applied to your workload environment.
  • Troubleshooting help: The Helm chart deploying the Smart Agent automatically applies labels OPTIMIZER_ID and OPTIMIZED_WORKLOAD_NAME to facilitate tracking and troubleshooting issues with the Application Resource Optimizer. The labels help you identify which objects are created by ARO and can be displayed with most kubectl get commands.
  • An ARO requirement change: The observed minimum of Kubernetes workload's pods over the previous seven days of metrics history increased from three to five.
23.12.14

OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.

October Release - 23.10.x 

Enhancements

CategorySummaryVersion
Alerting



You can now create a health rule condition for percentile values on a histogram metric. For example, for the apm:business_transaction entity type, you can create a health rule condition based on the single metric Response Time  - Histogram (ms). Currently, the following percentile values are supported: P50P75P90P95, and P99. The percentile based health rules reduce noise and increase the accuracy of alert generation.23.10.27

You can now create health rule conditions for events and get alerts for the abnormal events. You can also apply filters based on the event attributes. For example, you can create a health rule to trigger alert when the count of the k8s events with Severity IN [Severe, Warning] && Event Reason = "*Failed*" > 10 in the last 30 mins. On the entity centric page, you can view the Violating Events graph and Violation Details related to the event.

The Violating Events chart displays the following information:

  • The total number of events that occurred during the look back time period (Use data from last x minutes)
  • The trend of the event count
  • The threshold value and the event counts that are deviating from the threshold
23.10.27

In health rules, when you apply filters for the entity types and their associated metrics, the values of the filter attributes are automatically suggested if available. For example, if you select the entity type k8s:pod, you can view the:

  • Autosuggested values of the attribute k8s.cluster.name
  • Autosuggested values of the attribute severity for the single metric Native k8s event count (events)
23.10.27

Documentation

Cisco AppDynamics has documented the purge time-to-live (TTL) for each entity. For more information, see Kubernetes Entities. For other domains, see the ECP documentation for your specific entity.

23.10.27
Cloud Infrastructure and Troubleshooting 





Cisco Cloud Observability now supports monitoring:

23.10.27
23.10.27
Kubernetes and App Service Monitoring

In this release, the Orchestration Client was renamed to the Cisco AppDynamics Smart Agent. As part of this change, the fso-agent-mgmt-client section of the operator-values.yaml file was renamed to appdynamics-smartagent.

For any upgrade from 23.6.0 to >=23.9.0, ensure that the Smart Agent values are updated in the operators-values.yaml file. See Upgrade or Uninstall Kubernetes and App Service Monitoring.

23.10.27
This release introduces a new Kubernetes predefined health rule, K8s CronJob Health Rule to determine the failed cronjobs.23.10.27
You can now use Enable Additional Configurations section at Configure > Kubernetes and App Services to generate the configuration file with selected collectors and operating systems. 
When you download the configuration files for operator and the required collectors, the configuration file includes the inline documentation of the additional settings. See Install Kubernetes and App Service Monitoring

Logs





Cisco Cloud Observability now supports log collection from additional sources:

  • Amazon Elastic Load Balancing (Amazon ELB) service logs (includes ALB, CLB, NLB)
  • Amazon Virtual Private Cloud (Amazon VPC) service logs
  • Applications running on Amazon Elastic Container Service (Amazon ECS) 
  • Applications running on AWS Fargate 
  • Applications running on non-Kubernetes® Linux hosts, such as Amazon Elastic Compute Cloud (EC2) and bare-metal Linux machines
23.10.27

You can now deploy the Log Collector on non-Kubernetes hosts, such as bare metal EC2s. Hosts must be running on Linux. In this deployment, the Log Collector sends logs from the host directly to the Cisco AppDynamics common ingestion service (CIS).

23.10.27

A new option under Configure > Kubernetes and APM > Enable Additional Configurations generates a collectors-values.yaml file for you, which simplifies the deployment of the Log Collector onto your clusters. See Configure the Log Collector.

23.10.27

A new interface, Configure > Log Processing, allows you to specify server-side log parsing rules to extract fields from incoming log messages at the time of ingestion. By saving these rules on the server, you can standardize field names, improve search performance, and apply the same rules to new log sources. 

23.10.27

In Configure > Data Security, we've expanded options for specifying the scope of a data masking rule. Previously, you specified the scope as the value of the logFormat parameter in collectors-values.yaml. Now, you can specify rule scope by selecting a log attribute from the Rule scope pull-down menu, and specifying a value for that attribute. Masking rules that you created with the logFormat parameter still work, but are no longer editable. Best practice is to delete your existing data masking rules and recreate them using the Rule scope menu. See Mask Sensitive Data.

23.10.27
The Relevant Fields panel on the Logs page now displays all available relevant fields with distinct values, count, and percentage, instead of a hardcoded list of 5 fields. This panel provides the fastest and most convenient way to filter your view of log messages.
See Explore Logs.
23.10.27
Service-oriented Investigations




The time range selector now has more capabilities. You can select a default time such as the last 5 minutes or 1 hour, a recently used time range, or a custom time where you can select from absolute or relative time. See Understand the Observe UI.23.10.27
Favoriting Business Transactions are now only available for select users with appropriate permissions. The favorites and shortcut options will be hidden for users with the Observer role. See Business Transactions.23.10.27
With Business Transactions, you can now configure and visualize the Revenue Loss metric  to correlate performance issues to business impacts and perform segment analysis. See Business Transactions.23.10.27
You can now filter based on specific metrics. Depending on what filter you use, a new column for metrics appears in the list view. See Filters.23.10.27
The UI theme has been updated to include new features such as updated colors, flowmap capabilities (see all entity pages), and light/dark mode options.23.10.27
Troubleshooting documentation has been updated for users that cannot collect some traces due to the size being greater than the allowed limit. See Troubleshoot Application Performance Monitoring with OpenTelemetry.23.10.27

Spectro Cloud® Palette

The following Cisco Cloud Observability monitoring solutions can now be installed and/or configured using add-on packs in the Spectro Cloud® Palette user interface:

This alternate installation process does not require manually accessing the Kubernetes cluster or using Helm chart commands.

23.10.27

Deprecation Notice

The Cluster Collectors <23.10 are deprecated. Therefore, ensure to upgrade the collector to version >=23.10. The support to collectors <23.10 will end on January 30, 2024. 

Impact: After January 30, 2024, you will not be able to monitor the associated Kubernetes entities using the relationship pane unless you have upgraded the Cluster Collector to version >=23.10.

Resolved Issues

KeySummaryVersion

PRE-14329

Observe was not loading for Cisco Cloud Observability Tenants. This issue is now resolved.

23.10.27

ALERT-14324

A few health rules related to the k8s clusters were not being evaluated. This issue is now resolved.

23.10.27

DASHUI-4925

In the Health Violation(s) timeline, there was a mismatch in the violation status of a health rule when observed at different time range. For example, a health rule violation status was displayed as open when you select the time range as Last 1 Hour. But when you select the time range as Last 6 Hours, the same health rule violation was displayed as closed. This issue is now resolved.

23.10.27

XTNSBL-3875

Azure Collectors for Cisco Cloud Observability were missing relations from SQL Server to SQL DB. This issue is now resolved.

23.10.27

XTNSBL-3874

Azure Collectors for Cisco Cloud Observability were pulling metrics from Azure entities belonging to other resource groups along with the resources belonging to the configured resource group. This issue is now resolved. 

23.10.27
XTNSBL-3077AWS tags were not collected for ECS task or task definition. This issue is now resolved. 23.10.27

Known Issues and Limitations

CategorySummary

Customer Onboarding and Startup

Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

Logs

Logs for Amazon ECS entities don't appear on the entity-centric pages (ECPs). Workaround: Navigate to Explore > Logs to search for these logs. 

Module Enhancements

CategorySummaryVersion

Extend Cisco Cloud Observability Modules 

The Application Resource Optimizer (ARO) now allows you to override blockers when configuring the optimization of your workloads for testing purposes. 

23.10.27

Extend Cisco Cloud Observability Modules 

Cost Insights enables you to set budgets and monitor costs for infrastructure resources grouped by teams. You can view the current and monthly forecasted costs, the assigned teams, the budget history, and set health rules to be alerted when costs exceed the budget.

23.10.27

Extend Cisco Cloud Observability Modules 

Cisco Secure Application now allows you to create HTTP alerts. 23.10.27




OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.

September Release - 23.9.x 

Enhancements

CategorySummaryVersion
Accounts

New Cloud Native Application Observability customers sign in to their FSO Tenants through https://login.fso.cisco.com/.

Existing Cloud Native Application Observability customers will still continue to sign in to their FSO Tenants through https://login.appdynamics.com/.

Neither new nor existing Cloud Native Application Observability customers need to take any action, as they will automatically be directed to the appropriate login page. The user experience is the same whether you sign in through https://login.fso.cisco.com/ or https://login.appdynamics.com/.

23.9.27
App Root Cause Analysis using Anomaly Detection

Anomaly Detection is now enabled for the following Kubernetes entity types:

  • Cluster
  • Namespace
  • Workload
  • Pod

This feature automatically detects any anomalies that arise in these Kubernetes entity types.

You can also configure anomaly detection for the specified Kubernetes entity types to perform the following tasks:

  • Link HTTP request actions
  • Tune sensitivity level— high, medium, and low
  • Test the anomaly detection for the entities in staging or production environments
23.9.27
Cloud Infrastructure and Troubleshooting 



Cisco Cloud Observability now supports monitoring Azure Cosmos DB.

23.9.27

You can now use the Cisco AppDynamics Infrastructure Collector to monitor Amazon Elastic Container Services (ECS) entities running on Amazon Fargate or Elastic Compute Cloud (EC2). See Set up the Cisco AppDynamics Infrastructure Collector to Monitor AWS.

Cisco Cloud Observability now supports monitoring: 

23.9.27

Cisco Cloud Observability now supports monitoring:

23.9.27
Cloud Native Application Observability Plugins

A new version of the Grafana plugin is now available and it supports the Include All option toggle.

23.9.27
Kubernetes and App Service MonitoringYou can now view the pod readiness and liveness probe information in the Properties panel of PodsWorkloadsClusters, and Namespaces. To view this data, you must upgrade the collectors to the latest version.23.9.27
Collectors and Operators

This release includes new versions of the following AppDynamics artifacts: 

  • OTel Docker image:
    • Linux: 23.9.0-1125
    • Windows: 23.9.0-1125-windows-amd64-nanoserver-ltsc2019
  • Cluster Collector Docker image: 23.9.0-1758
  • Infrastructure Collectors Docker image: 23.9.0-4856
  • AppDynamics Operator Docker image: 23.9.0-1101
  • AppDynamics Collectors Helm chart: 1.14.714
  • AppDynamics Operator Helm chart:  1.15.174
23.9.27
Logs

On the Logs page, pattern detection is now available for containers (previously was only available for service instances), and the Log Pattern tab is now Patterns.

23.9.27

Observability


On the Observe page for an entity, you now get an alert message when some of the health rules are in the unknown state. Expand the Health Violations section to view the timelines of the unknown health rules. When you click an unknown health rule timeline, you can view the reason for its unknown status.23.9.27
On the Observe page for an entity, you can now view the Violating Metrics chart for a metric expression associated with a health rule. Click the health rule from the Health Violations time line to view the Violating Metrics chart.23.9.27

When you create an HTTP request action, you can now specify the timestamp format for the attributes event_time, violation_start_time, and violation_end_time. You can view the timestamp in one the following formats:

  • Date and time format
  • Epoch format
  • A valid custom format such as yyyy-MM-dd HH:mm:ss
23.9.27
Service-oriented Investigations

You can now differentiate between Lambda and APM domains, filter by Lambda entities, and view Lambda details in the unified service detail view and via the Properties panel. See  Services.

23.9.27

Resolved Issues

KeySummaryVersion
DASHUI-4323

Log details were not displayed when you navigate to Observe > Kubernetes > Hosts > <Host Name> > View All Logs. This issue occurred when Log Collector is deployed on Google Kubernetes Engine (GKE). Cisco AppDynamics has released the support of GKE and the issue has now been resolved. 

23.9.0

The Log Collector did not filter or scrub out any sensitive data present in log messages. This issue is now resolved.23.9.0

On the Logs page, the Log Patterns tab did not not appear when you used regular browser mode. This issue has now been resolved.

23.9.0

The filter bar misinterpreted the word Severity because it's a reserved word. This issue has now been resolved.23.9.0

The list view could only display a maximum of 1000 entities of one type. This issue has now been resolved.

23.9.0

On Clusters and Namespaces pages, the Persistent Volume Claim (PVC) events are not displayed in Kubernetes Events > View Event Summary. This issue is now resolved. 

23.9.0

Cisco Cloud Observability did not support more granular access control. Now, Cisco Cloud Observability offers granular control and users can be assigned other roles including custom roles as needed.

23.9.0

Known Issues and Limitations

CategorySummary

Customer Onboarding and Startup

Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

Modules 

Enhancements

CategorySummaryVersion

Extend Cisco Cloud Observability Modules 

Cost Insights now provides the ability to create Teams that are associated with a domain such as Kubernetes, APM, or infrastructure. Using Teams, you can view the aggregated costs of infrastructure across a domain.

23.9.27

OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.

August Release - 23.8.x 

Enhancements

CategorySummaryVersion
App Root Cause Analysis using Anomaly DetectionOn the Health Violation timeline of an entity, when you click the Anomaly Detection event, the call paths of the anomaly are now highlighted on the Flow map. This feature is currently applicable for the entities related to Business Transactions and Services. On the Flow map, you can easily trace the call paths and examine the suspected causes of the anomaly. When you click Call Path under Suspected Causes, the corresponding call path is highlighted.23.8.23
Business Metrics for Business Transactions

With Business Transactions, you can now configure and visualize metrics such as Average Cart Value, Total Carts Sold, and Total Cart Value to correlate performance issues to business impacts and perform segment analysis. See Business Transactions. Business Metrics are only available if you are a part of the Early Access Program (EAP). For more information, contact your Cisco AppDynamics administrator or support team. 

23.8.23
Cloud Infrastructure and Troubleshooting 

Cisco Cloud Observability now supports monitoring Azure Database for MariaDB (Single server).

23.8.23
23.8.23

Cisco Cloud Observability now supports monitoring AWS Database Migration Service (DMS).

23.8.23

Cisco Cloud Observability now supports monitoring Google Cloud Platform App Engine.

23.8.23

Version 2.0.2 of the Cisco Cloud Observability Connections API is released. This release introduces support for Google Cloud Platform. See Cloud Native Application Observability Connections API.

23.8.23
Collectors and OperatorsThe AppDynamics Collectors Helm chart is updated to 1.13.658.23.8.28

The following artifacts are updated:

  • Orchestration Client Docker image: 23.8.0-721
  • AppDynamics Operator Helm chart: 1.14.168
23.8.24

This release includes new versions of the following AppDynamics artifacts: 

  • OTel Docker image:
    • Linux: 23.8.0-1068
    • Windows: 23.8.0-1068-windows-amd64-nanoserver-ltsc2019
  • Orchestration Client Docker image: 23.8.0-675
  • Cluster and Infrastructure Collectors Docker image: 23.8.0-1713
  • AppDynamics Operator Docker image: 23.8.0-1088
  • AppDynamics Collectors Helm chart: 1.13.648
  • AppDynamics Operator Helm chart:  1.14.156
23.8.23
Kubernetes® and App Service Monitoring
  • You can now view when a pod was created for the Kubernetes entities from the entity list view using the Created At column.
  • The data for following Kubernetes entities are retained for three weeks after the entity becomes inactive:
    • k8s:managed_job
    • k8s:pod
    • infra:container
23.8.23
Log Collection
  • The Log Collector exports additional self-telemetry metrics from Filebeat, and deprecates others. The Filebeat metrics exported by default are: 
    • beat.memstats.memory_alloc
    • filebeat.events.active
    • filebeat.input.filestream.events.create
    • filebeat.input.filestream.events.delete
    • filebeat.input.filestream.events.eof
    • filebeat.input.filestream.events.rename
    • filebeat.input.filestream.events.truncate
    • filebeat.input.filestream.events.write
    • filebeat.input.filestream.files.open
    • filebeat.input.filestream.harvester.running
    • filebeat.input.filestream.harvester.stopped
    • libbeat.output.read.errors
    • libbeat.output.write.bytes
    • libbeat.output.write.errors
    • system.load.norm.5
    • system.load.norm.15
  • The Log Collector now supports deployment on Google Kubernetes Engine (GKE) platform.
23.8.23

Service-oriented Investigations

You are now able to share data with other Cloud Native Application Observability users. The Group View, Filter View, and Time Range values remain the same when shared. See Understand the Observe UI.

23.8.23
Now, when single-clicking on an entity (node or interaction), the selected node or interaction will be highlighted in blue.23.8.23

Resolved Issues

None.

Known Issues and Limitations

Customer Onboarding and Startup

  • Every user has the ability to configure and view all data. This release does not support more granular access control.
  • Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

  • Cisco Cloud Observability is available in AWS US West, AWS Sydney and AWS Frankfurt regions only. See About Cisco Cloud Observability and SaaS Domains and IP Ranges

  • For the HostProcess containers, the Cisco AppDynamics Collectors do not collect metrics from Windows Exporter. This happens because Windows HostProcess container monitoring is not supported. 

Kubernetes Infrastructure and Troubleshooting

  • On Clusters and Namespaces pages, the Persistent Volume Claim (PVC) events are not displayed in Kubernetes Events > View Event Summary. However, these events are listed in the Events detail view.
    To view the PVC events, click Kubernetes Events > Show All Events
  • The list view displays a maximum of 1000 entities of one type.
  • The Properties panel shows limited metadata.
  • If the cluster you connect to Cisco Cloud Observability already has Cisco AppDynamics APM agents running on it, the metrics you see on the Observe page may be different from the metrics you see on your Cisco AppDynamics APM Controller. Comparing metrics on the two platforms may yield inconclusive results. There are multiple reasons for this. Cisco Cloud Observability only ingests metrics from AWS CloudWatch, Azure Monitor, and OpenTelemetry™-compatible collectors, whereas Cisco AppDynamics App Server agents, such as Machine Agent, send metrics in a proprietary, non-OpenTelemetry format. In addition, those agents are connected to your Controller UI, not to Cisco Cloud Observability. 

  • Some metrics are not displayed for ingress and service entities. Cisco Cloud Observability only displays metrics if your ingress controller is the Kubernetes NGINX controller. There is no support for displaying metrics from other ingress controllers at this time. In addition, if your ingress controller is configured with wildcard domains, there are no metrics for that ingress. To get metrics in this case you need to configure the ingress controller with --metrics-per-host=false (you will lose labeling by hostname, but still have labeling by ingress). 

Service-oriented Investigations

No known issues.

App Root Cause Analysis using Anomaly Detection

No known issues.

Application Troubleshooting with Logs

  • Log details are not displayed when you navigate to Observe > Kubernetes > Hosts > <Host Name> > View All Logs. This issue occurs when Log Collector is deployed on Google Kubernetes Engine.
    Workaround: To view the logs for a host on a Google Cloud Platform, you require to navigate to Explore > Logs > Add Filter, and then add filters for attributes(host.id) or attributes(host.name).

  • The Log Collector does not filter or scrub out any sensitive data present in log messages. However, you can configure it to drop (omit) fields you specify. See Log Collector Settings
  • On the Logs page, the Log Patterns tab does not appear when you use regular browser mode. Workaround: Use a browser in incognito mode. 
  • The filter bar misinterprets the word Severity because it's a reserved word. Avoid this word in full-text searches.

Users may see the 'No Data' message due to query timeout when navigating from the Business Transaction Security tile of the Inspector panel to see log entries with critical sensitive data, especially when the time range selected is 1 Week or more. Workaround: Select a shorter custom time range on the Business Transaction page and try again.

Modules 

Enhancements

CategorySummaryVersion

Extend Cisco Cloud ObservabilityModules 

Cisco AppDynamics is introducing Cisco Secure Application, available through the FSO Platform Exchange on September 12th, 2023. Cisco Secure Application helps secure cloud-native applications based on real-time vulnerability analytics, and business risk observability. 

23.8.23

Extend Cisco Cloud ObservabilityModules 

The Application Resource Optimizer is generally available for 23.8 and has improved the UI with additional guardrails and more detailed information and instructions about blockers preventing the optimization of your Kubernetes workloads.

23.8.23




OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.

July Release - 23.7.x 

Enhancements

CategorySummaryVersion
Cloud Infrastructure and Troubleshooting

 Cisco Cloud Observability now supports monitoring the following Google Cloud Platform (GCP) entities:

23.7.27
Cloud Infrastructure and Troubleshooting

Cisco Cloud Observability now supports monitoring the following Amazon entities:

23.7.27
Cloud Infrastructure and Troubleshooting

Cisco Cloud Observability now supports monitoring Azure Database for MySQL (Flexible and Single server). See Azure Database Services.

23.7.27
Collectors and Operators

This release includes new versions of the following AppDynamics artifacts: 

  • OTel Docker image:
    • Linux: 23.7.0-1005
    • Windows: 23.7.0-1005-windows-amd64-nanoserver-ltsc2019
  • Cluster and Infrastructure Collectors Docker image: 23.7.0-1679
  • AppDynamics operator Docker image: 23.7.0-1078
  • AppDynamics Collectors Helm chart: 1.12.622
  • AppDynamics Operator Helm chart:  1.13.146
23.7.27
Kubernetes and App Service Monitoring
The Kubernetes and App Service Monitoring is now supported on GKE 1.26. However, Log Collectors are not yet supported on the GKE platform.
23.7.27
Application Performance Monitoring with OpenTelemetry™


Cisco Cloud Observability now supports supports instrumenting and monitoring AWS Lambda using OpenTelemetry layers. See:

23.7.27
Application Performance Monitoring with OpenTelemetry™

Cisco AppDynamics has documented how to Troubleshoot Application Performance Monitoring.

23.7.27
Business Transactions



You can now create Favorited Business Transactions (BTs) and user defined BTs. See:

With the ability to create your own BTs and favorite those that are most important, we are removing the list filter based on calls/min to give you access to more BTs.

23.7.27
Tags
There is now updated information about tags and tag management services. See Tags.
23.7.27
FSO Platform Modules

The Application Resource Optimizer has the following enhancements: 

  • Support for light theme  
  • Improved formatting for discovered service level objective (SLO) values 
23.7.27
FSO Platform Modules
The Kubernetes Workload and Cost Profiles detects and warns about the missing Orchestration Client.
23.7.27
Log Collector



  • Using our AWS CloudFormation template, you can deploy a CloudFormation stack to forward CloudWatch logs from your AWS services to the AppDynamics Common Ingestion Service, where they are associated with the right entities in your observability domains. The entire deployment is described in a CloudFormation template we provide. Currently, the template only supports onboarding of EC2 and Lambda logs. 

  • You can now onboard logs through OpenTelemetry. There are two ways to do this: through the new filelog receiver in otel-collector or through the otel-java-agent. Both ways require changes to the Kubernetes and App Service Monitoring configuration parameters. Documentation is not yet available for this. For help, contact the support team.
  • The latest Helm chart includes a new parameter which prevents sequential rolling restarts of all Log Collector pods whenever you change its configuration. Sequential rolling restarts take a long time on large clusters. To get this change, download the latest Helm chart and Cisco AppDynamics Operator, and add the parameter logCollectorPod.rollingUpdateMaxUnavailable to your collectors-values.yaml.

  • The Log Collector does not support deployment onto Google Kubernetes Engine (GKE). If you deploy Kubernetes and App Service Monitoring onto GKE, you won't be able to deploy the Log Collector at this time.

  • True default log collection is now supported. New deployments don't need to change anything in collectors-values.yaml for this feature; you can simply deploy the Log Collector and see your logs in the system immediately, parsed by timestamp only. You can then incrementally refine your parsing configurations.

  • You can specify a condition for excluding log collection by setting the new parameter logCollectorConfig.container.excludeCondition in your collectors-values.yaml.

  • All existing Log Collector deployments must update to the latest Helm chart. See Upgrade or Uninstall Database Monitoring.
23.7.27
Logs Page
The ability to view surrounding log messages (messages immediately before and after a target message) turbo charges your troubleshooting by giving you insight into root causes and what lead to the initial failure.
23.7.27

Query Builder


There's a new page for exploring MELT data,  Explore > Query Builder, which provides a way to send Unified Query Language (UQL) requests from the user interface. Previously, you could only send UQL request through the UQL API. Adding query support to the user interface allows for seamless exploration of data and enhances your ability to discover the "unknown unknowns" in your ecosystem. The Query Builder is connected to the Schema Browser, which gives you a jumpstart on constructing queries. Together, these two new tools are the first phase in our enhanced support for exploring the "unknown unknowns".

23.7.27

Schema Browser


You can now use Explore > Schema Browser to view the entity types and their associations across the domains in your full stack. It displays a graphical representation of your schema and enables you to explore the various entity types and their metadata. You can also query metadata without having knowledge of the Unified Query Language (UQL). See Schema Browser.

23.7.27


Monitor Health


When you create HTTP Request Actions, you can now select one of the following action templates:

  • OpsGenie
  • PagerDuty
  • ServiceNow
  • Slack
  • Webex

These templates helps you by automatically pre-filling some of the application-specific attributes such as raw URL, custom variables, and payload body. Therefore, you can quickly create an HTTP request action for these specified applications. See Create HTTP Request Action.

23.7.27


Monitor Health


While specifying a health rule condition, you can now add filters on the attributes of a single metric and monitor the entity. The Metric attribute filters (Optional) option is only visible for a metric that has attributes. For example, you can monitor an entity and trigger alert when the metric of span count with http.status_code = 400 is greater than 5. See Define Health Rule Evaluation Conditions.
23.7.27


App Root Cause Analysis using Anomaly Detection


You can now configure anomaly detection for the Infrastructure entity types- AWS Application Load Balancer and AWS Classic Load Balancer and link an action that gets triggered for an anomaly. The anomaly detection algorithm automatically detects anomalies that originate in the infrastructure and propagate to business transactions and service instances.

The following metrics are monitored for AWS Application Load Balancer:

  • Backend/ Target Response Time
  • Backend/ Target Connection Errors

The following metrics are monitored for AWS Classic Load Balancer:

  • Backend/ Target Response Time
  • Backend/ Target Connection Errors
  • Surge Queue Length

See Configure Anomaly Detection.

23.7.27
Cloud Native Application Observability Plugins

The Cisco Cloud Observability Grafana Plugin 23.1.0-28 is available on the Downloads portal. The following changes were implemented:

  • Bug fix for a Grafana change that caused some panels to show No Data. The default dashboards will need to be re-imported.
  • Internal API change that requires customers to update their plugins. After the July release, there can be indeterministic issues with data loading.

You can now upgrade the Cisco Cloud Observability Grafana plugin. See Upgrade the Grafana Plugin.

23.7.27
FSO Platform API

The FSO Access Management API allow an administrator (or any user with sufficient privileges) to manage access for their users. See Access Management API.

23.7.31

Resolved Issues

KeySummaryVersion

COGENG-6656

The long running AD Alarm with same severity, was not publishing the AD Events to the Dash base. This issue is now resolved and AD Events are visible in the UI.23.7.0

ALERT-13374

Duplicate entries for child types in EvaluationObjectHierarchy#childrenMap were being displayed. This issue is now resolved.

23.7.0

ALERT-14088

The actions metadata service was deleting the health rule handler. This issue is now resolved.23.7.0

Known Issues and Limitations

Customer Onboarding and Startup

  • Every user has the ability to configure and view all data. This release does not support more granular access control.
  • Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

  • Cisco Cloud Observability is available in AWS US West, AWS Sydney and AWS Frankfurt regions only. See About Cisco Cloud Observability and SaaS Domains and IP Ranges

  • For the HostProcess containers, the Cisco AppDynamics Collectors do not collect metrics from Windows Exporter. This happens because Windows HostProcess container monitoring is not supported. 

     

Kubernetes Infrastructure and Troubleshooting

  • On Clusters and Namespaces pages, the Persistent Volume Claim (PVC) events are not displayed in Kubernetes Events > View Event Summary. However, these events are listed in the Events detail view.
    To view the PVC events, click Kubernetes Events > Show All Events.
  • Cisco Cloud Observability may contain links to documentation for Cisco AppDynamics APM, which is a completely different product. Workaround: Use only the documentation on https://docs.appdynamics.com/appd-cloud
  • The list view displays a maximum of 1000 entities of one type.
  • The Properties panel shows limited metadata.
  • If the cluster you connect to Cisco Cloud Observability already has Cisco AppDynamics APM agents running on it, the metrics you see on the Observe page may be different from the metrics you see on your Cisco AppDynamics APM Controller. Comparing metrics on the two platforms may yield inconclusive results. There are multiple reasons for this. Cisco Cloud Observability only ingests metrics from AWS CloudWatch, Azure Monitor, and OpenTelemetry™-compatible collectors, whereas Cisco AppDynamics App Server agents, such as Machine Agent, send metrics in a proprietary, non-OpenTelemetry format. In addition, those agents are connected to your Controller UI, not to Cisco Cloud Observability. 


  • Some metrics are not displayed for ingress and service entities. Cisco Cloud Observability only displays metrics if your ingress controller is the Kubernetes NGINX controller. There is no support for displaying metrics from other ingress controllers at this time. In addition, if your ingress controller is configured with wildcard domains, there are no metrics for that ingress. To get metrics in this case you need to configure the ingress controller with --metrics-per-host=false (you will lose labeling by hostname, but still have labeling by ingress). 

Service-oriented Investigations

No known issues.

App Root Cause Analysis using Anomaly Detection

No known issues.

Application Troubleshooting with Logs

  • The Log Collector does not filter or scrub out any sensitive data present in log messages. However, you can configure it to drop (omit) fields you specify. See Log Collector Settings
  • On the Logs page, the Log Patterns tab does not appear when you use regular browser mode. Workaround: Use a browser in incognito mode. 
  • The filter bar misinterprets the word Severity because it's a reserved word. Avoid this word in full-text searches.



OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.

June Release - 23.6.x 

Enhancements

CategorySummaryVersion
Service-oriented Investigations

The autocomplete feature is now incorporated for all Traces, Spans, and topology fields. Fields that are related to Traces or Spans now displays the count value. See Traces or Spans.

23.6.28
Service-oriented InvestigationsFor the entity Service Instances, there are two new additional columns that display JMX metrics (Heap % and System CPU %). Also, RPC Backend data information is now available. See Service Instances Detail.23.6.28
Cloud Infrastructure and Troubleshooting

Cisco AppDynamics now supports monitoring Google Cloud Platform services. See Configure Google Cloud Platform Connection.

This release supports GCP Compute Engine Instance.

23.6.0
Cloud Infrastructure and Troubleshooting

Cisco AppDynamics now supports monitoring Amazon API Gateway.

23.6.0
Application Performance Monitoring with OpenTelemetry

Cisco AppDynamics has documented the following configuration options for the Cisco AppDynamics Distribution of OpenTelemetry Collector:

23.6.0
Application Performance Monitoring with OpenTelemetry

Two new configuration options, annotations and labels, have been added to the Cisco AppDynamics Distribution of OpenTelemetry Collector. You can use these configuration options to add or override annotations and labels. See Advanced Settings for the Cisco AppDynamics Distribution of OpenTelemetry Collector.

23.6.0
Kubernetes and App Service Monitoring

This release includes new versions of the following AppDynamics artifacts: 

  • OTel Docker image:
    • Linux: 23.6.0-945
    • Windows: 23.6.0-945-windows-amd64-nanoserver-ltsc2019
  • Cluster and Infrastructure Collectors Docker image: 23.6.0-1659
  • AppDynamics operator Docker image: 23.6.0-1052
  • AppDynamics Collectors Helm chart: 1.11.589
  • AppDynamics Operator Helm chart:  1.12.144
  • DB Collector Docker Image 23.6.0-780
23.6.0
Kubernetes and App Service MonitoringThis release introduces the Kubernetes Configurations (ConfigMaps and Secrets) monitoring with a unique feature of getting real time insights to modified ConfigMaps and Secrets. See Configurations.23.6.0
Kubernetes and App Service Monitoring

You can now observe the details of the Horizontal Pod Autoscaling on the Workloads page. See Autoscalers.

23.6.0
Kubernetes and App Service Monitoring

Cisco AppDynamics now supports the Orchestration Client, which enables remote agent management (orchestration and configuration) for Kubernetes-related agents. See:

This release supports remotely managing the life cycle of the Application Resource Optimizer agent. See Application Resource Optimizer (ARO).

23.6.0-283


Extend Cisco AppDynamics

There is now end-user documentation a part of Cisco Cloud Observability. You can build and discover Cisco Cloud Observability Modules and applications on the FSO Platform Exchange. You can also view Cisco Cloud Observability Plugins. See:

23.6.28

Extend Cisco AppDynamics

The following three modules built on the Full-Stack Observability (FSO) Platform extend and enhance Cisco AppDynamics are now available through the FSO Platform Exchange:

  • Cost Insights -  leverages the existing customer metrics of the FSO platform, analyzes the workloads running in cloud and cloud-native infrastructures, and generates detailed cost data for all infrastructure workloads and services. 
  • Kubernetes Cost and Workload Profiler - analyzes workloads running on a Kubernetes cluster to identify performance and reliability risks and inefficient configurations.
23.6.28
Database Monitoring

This release supports the following databases:

  • MySQL
  • MariaDB
  • Aurora MySQL

See Supported Databases.

23.6.0
Alert and Respond

On the Observe page, the Health and Alerting section now displays the following violation details in an improved format for better readability:

  • Health rule name
  • Violating metric
  • Source of the metric
  • Metric value that caused the violation
  • Condition for the violation (applicable for a health rule)
  • Start time of the violation
  • Status (Open or Close) of the violation
  • Suspected causes (applicable for anomaly detection)
  • Call paths (applicable for anomaly detection)
23.6.0
Alert and RespondIn the Create New Health Rule wizard, you can now use the not equal to (!=) operator in the filter expression for the monitored entities. See Define Entity Details.23.6.0
Application Troubleshooting with Logs

Log Collector updates:

  • Use the new page, Configure > Data Security, to create expressions and rules for masking sensitive data in log messages. Automatic redaction, which happens after logs are ingested but before logs are indexed and stored, offers an extra layer of protection. See Mask Sensitive Data.
  • We've added a new configuration parameter to collectors-values.yaml, logFormat in simplified YAMLs or appd.log.format in advanced YAMLs. This parameter specifies the log "scope". This parameter is required for sensitive data masking. ​See Log Collector Settings or Log Collector Settings - Advanced YAML Layout.
  • Breaking changes:
    • In simplified YAML configuration files, you must update all fields that use a regular expression (regex) value. Change all double escapes (three backslashes) to single escapes (one backslash).
      For example, change

      "[%date{\\\"yyyy-MM-dd'T'HH:mm:ss,SSS\\\"}Z] [%-5level] [%thread] [%logger{36}] %msg%n"
      YML

      to

      "[%date{\"yyyy-MM-dd'T'HH:mm:ss,SSS\"}Z] [%-5level] [%thread] [%logger{36}] %msg%n"
      YML


      Impacted fields include all message patterns, all sub parser patterns, all multi parser patterns, and the timestampField field in JSON parser configurations.

      The exception to this rule is for patterns that span multiple lines (using |- or > or similar YAML operators). For example, you don't need to change the value of pattern here:

      messageParser:
        multi:
          enabled: true
          pattern: |-
            {
              …
            }
      YML
    • Change all multi-line patterns from double quotes to single quotes, and double escapes to single escapes. For example: 

      '^\[otel|^\[[0-9]{4}'
      YML
  • To associate logs with your workload entities have workload names appear in the list of relevant fields in the left panel of the Logs page, enrich them with the k8s.workload.name attribute. This feature is only available for Log Collector deployments that use the advanced YAML layout. See Log Collector Settings - Advanced YAML Layout.
23.6.28

Resolved Issues

KeySummaryVersion
OTEL-464Users were unable to specify the clusterId manually if the helm lookup function was not available. This issue is now resolved as the clusterId parameter using helm values is now exposed.23.6.28
LCA-364Users can now singly escape all the values inside quotes as expected and can avoid any escaping in the multiline values.23.6.28

Known Issues and Limitations

Customer Onboarding and Startup

  • Every user has the ability to configure and view all data. This release does not support more granular access control.
  • Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

  • Cisco Cloud Observability is available in AWS US West, AWS Sydney and AWS Frankfurt regions only. See About Cisco Cloud Observability and SaaS Domains and IP Ranges

  • For the HostProcess containers, the AppDynamics Collectors do not collect metrics from Windows Exporter. This happens because Windows HostProcess container monitoring is not supported. 

Kubernetes Infrastructure and Troubleshooting

  • On Clusters and Namespaces pages, the Persistent Volume Claim (PVC) events are not displayed in Kubernetes Events > View Event Summary. However, these events are listed in the Events detail view.
    To view the PVC events, click Kubernetes Events > Show All Events.
  • Cisco Cloud Observability may contain links to documentation for AppDynamics APM, which is a completely different product. Workaround: Use only the documentation on https://docs.appdynamics.com/appd-cloud
  • The list view displays a maximum of 1000 entities of one type.
  • The Properties panel shows limited metadata.
  • If the cluster you connect to Cisco Cloud Observability already has AppDynamics APM agents running on it, the metrics you see on the Observe page may be different from the metrics you see on your AppDynamics APM Controller. Comparing metrics on the two platforms may yield inconclusive results. There are multiple reasons for this. Cisco Cloud Observability only ingests metrics from AWS CloudWatch, Azure Monitor, and OpenTelemetry™-compatible collectors, whereas AppDynamics App Server agents, such as Machine Agent, send metrics in a proprietary, non-OpenTelemetry format. In addition, those agents are connected to your Controller UI, not to Cisco Cloud Observability. 

  • Some metrics are not displayed for ingress and service entities. Cisco Cloud Observability only displays metrics if your ingress controller is the Kubernetes NGINX controller. There is no support for displaying metrics from other ingress controllers at this time. In addition, if your ingress controller is configured with wildcard domains, there are no metrics for that ingress. To get metrics in this case you need to configure the ingress controller with --metrics-per-host=false (you will lose labeling by hostname, but still have labeling by ingress). 

Service-oriented Investigations

No known issues.

App Root Cause Analysis using Anomaly Detection

No known issues.

Application Troubleshooting with Logs

  • The Log Collector does not filter or scrub out any sensitive data present in log messages. However, you can configure it to drop (omit) fields you specify. See Log Collector Settings

OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.

May Release - 23.5.x 

Enhancements

CategorySummaryVersion
Application Troubleshooting with Logs
  • Properties panel improvements: Some fields or attributes like k8s.cluster.id, k8s.cluster.name, k8s.pod.name, fields.k8s.workload.name, host.name, host.id, and traceId have hyperlinks. When you click a hyperlinked field and then click View Details you navigate to the associated entity centric page.
  • Filter bar enhancements: Full-text based search for logs is now supported. You can use full-text search without the old syntax of `raw = '<token>'`. The filter bar interprets your full-text entries with AND logic—in other words, all tokens must be present.
    • To filter based on attributes, click the "+" button below the filter bar. To clear a filter, click its "x" button.
    • To filter by severity, create a filter for the attribute named level. The legacy way to filter by severity using the Severity pull-down menu is deprecated and will be removed soon.  
  • Relevant fields: A new panel on the left displays the relevant fields with their top values for you to quickly add to or exclude from a filter. 
  • Trace-to-log and log-to-trace correlation: You can now view log messages related to a trace by clicking Associated Logs on the Traces page. From the Logs page, you can select a trace or span ID in the Properties panel, then click View Details to see the trace related to that log message. For maximum benefit from this feature, instrument your log messages with trace and span IDs.
23.5.25
App Root Cause Analysis using Anomaly DetectionOn the Violating Metrics graph, you can now view the upper and lower bounds (threshold) of a metric when an anomaly is detected for an entity. The upper and lower bounds define the normal range of a metric. From the graph, you can identify the metric values that are outside the upper and lower bounds. See Monitor Anomalies.23.5.25
Kubernetes Infrastructure and Troubleshooting

This release includes new versions of the following AppDynamics artifacts: 

  • OTel Docker image:
    • Linux: 23.5.0-925
    • Windows: 23.5.0-925-windows-amd64-nanoserver-ltsc2019
  • Cluster and Infrastructure Collectors Docker image: 23.5.0-1593
  • AppDynamics operator Docker image: 23.5.0-1029
  • AppDynamics Collectors Helm chart: 1.10.540
  • AppDynamics Operator Helm chart:  1.11.124 
23.5.25
Service-oriented Investigations

Business Transactions (BTs) are now available on Cisco Cloud Observability. A Business Transaction is a visual representation of the transactions (request and response) that take place for the services on the application. BTs give a more granular view of services and interactions based on its starting point. It represents the processing path that is followed in response to a user-initiated request–in other words, the logical path of a user through your services. See: 

23.5.25
Kubernetes Infrastructure and Troubleshooting

AppDynamics now supports the following platforms in a Linux environment:

  • Rancher Kubernetes Engine (RKE) 1.25.9
  • Rancher Kubernetes Engine Government (RKE2) 1.24.13+rke2r1 
  • Tanzu Kubernetes Grid Integrated Edition (TKGI) 1.25.4+vmware.1

See Install Kubernetes and App Service Monitoring.

23.5.25

Resolved Issues

KeySummaryVersion
O2-2500Users can now specify clusterId manually if the Helm lookup function is not available. 23.5.25

UQL-4452

There was an issue where events could not be retrieved for display. Due to a configuration change in query service, this issue is now resolved.

23.5.25

Known Issues and Limitations

Customer Onboarding and Startup

  • Every user has the ability to configure and view all data. This release does not support more granular access control.
  • Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

  • Cisco Cloud Observability is available in AWS US West, AWS Sydney and AWS Frankfurt regions only. See About Cisco Cloud Observability and SaaS Domains and IP Ranges

  • For the HostProcess containers, the AppDynamics Collectors do not collect metrics from Windows Exporter. This happens because Windows HostProcess container monitoring is not supported. 

     

Kubernetes Infrastructure and Troubleshooting

  • On Clusters and Namespaces pages, the Persistent Volume Claim (PVC) events are not displayed in Kubernetes Events > View Event Summary. However, these events are listed in the Events detail view.
    To view the PVC events, click Kubernetes Events > Show All Events.
  • Cisco Cloud Observability may contain links to documentation for AppDynamics APM, which is a completely different product. Workaround: Use only the documentation on https://docs.appdynamics.com/appd-cloud
  • The list view displays a maximum of 1000 entities of one type.
  • The Properties panel shows limited metadata.
  • If the cluster you connect to Cisco Cloud Observability already has AppDynamics APM agents running on it, the metrics you see on the Observe page may be different from the metrics you see on your AppDynamics APM Controller. Comparing metrics on the two platforms may yield inconclusive results. There are multiple reasons for this. Cisco Cloud Observability only ingests metrics from AWS CloudWatch, Azure Monitor, and OpenTelemetry™-compatible collectors, whereas AppDynamics App Server agents, such as Machine Agent, send metrics in a proprietary, non-OpenTelemetry format. In addition, those agents are connected to your Controller UI, not to Cisco Cloud Observability. 

  • Some metrics are not displayed for ingress and service entities. Cisco Cloud Observability only displays metrics if your ingress controller is the Kubernetes NGINX controller. There is no support for displaying metrics from other ingress controllers at this time. In addition, if your ingress controller is configured with wildcard domains, there are no metrics for that ingress. To get metrics in this case you need to configure the ingress controller with --metrics-per-host=false (you will lose labeling by hostname, but still have labeling by ingress). 

Service-oriented Investigations

No known issues.

App Root Cause Analysis using Anomaly Detection

No known issues.

Application Troubleshooting with Logs

  • The Log Collector does not filter or scrub out any sensitive data present in log messages. However, you can configure it to drop (omit) fields you specify. See Log Collector Settings
  • On the Logs page, the Log Patterns tab does not appear when you use regular browser mode. Workaround: Use a browser in incognito mode. 
  • When you navigate directly to the Logs page from Explore > Logs, some fields or attributes in the Properties panel are not hyperlinked back to their associated entities. Workaround: Navigate to the Logs page from the entity's detail view ("entity centric page").

  • The filter bar misinterprets the word Severity because it's a reserved word. Avoid this word in full-text searches. Also, when you select a recent filter, the Severity menu quick filter doesn't reset to the value you used in that recent filter. 

OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.

April Release - 23.4.x 

Enhancements

CategorySummaryVersion
Kubernetes Infrastructure and TroubleshootingThis release includes AppDynamics Collectors Helm chart version 1.9.533.23.4.1
Kubernetes Infrastructure and Troubleshooting

A new global.clusterId parameter is introduced, which is required only if helm lookup is unsupported such as, when using Argo CD, helm template, or helm install --dry-run. See Configure Cisco AppDynamics Kubernetes Collectors.

23.4.1
Cloud Infrastructure and Troubleshooting

You can now monitor the following Azure entities: 

23.4.0
Application Troubleshooting with Logs

Log Collector improvements:

  • The simplified YAML configuration file now supports separate override sections for Windows and Linux operating systems.
  • The advanced YAML configuration file has renamed a parameter to reflect new OTLP exporter options: output.otlplog is now output.otlploggrpc.
23.4.0
Application Troubleshooting with Logs

The Logs page is updated with:

  • A reorganized Fields panel with frequently used fields (raw message and severity) at the top, and a button to wrap or unwrap the raw message field.
  • Linking back to the entity-centric page from some attributes: Select a log message, and in the Fields panel, click its k8s.cluster.idk82.cluster.name, or k8s.pod.name attribute to go to the corresponding entity's detail view.
  • Support for time selection on Histogram by drag-selecting an area without changing the global time range.
23.4.0
App Root Cause Analysis using Anomaly Detection

You can now configure anomaly detection for the AWS EC2 entity type and link an action that gets triggered for an anomaly. The anomaly detection algorithm automatically detects anomalies that originate in the infrastructure and propagate to business transactions and service instances. The following metrics are monitored for AWS EC2:

  • CPU used utilization
  • Memory used utilization
  • Page faults per second
  • Network incoming packets dropped
  • Network incoming errors per minute
  • Network outgoing packets dropped
  • Network outgoing errors per minute
  • Disk average IO utilization

See Configure Anomaly Detection.

23.4.0
Kubernetes Infrastructure and Troubleshooting

This release includes new versions of the following AppDynamics artifacts:

  • OTel Docker image: 23.4.0-888 (Windows)
  • Cluster and Infrastructure Collectors Docker image: 23.4.0-1530
  • AppDynamics operator Docker image: 23.4.0-1005
  • Log Collector Docker image: 23.4.0-567
  • AppDynamics Collectors Helm chart: 1.9.526
  • AppDynamics Operator Helm chart:  1.10.122
23.4.0
Kubernetes Infrastructure and TroubleshootingThe appearance of the Events page is updated to align with the appearance of the Logs page.23.4.0
Kubernetes Infrastructure and Troubleshooting

A new attribute, k8s.cluster.id, is introduced to ensure cluster uniqueness and avoid collision of data within a tenant that has the same cluster name. However, this update affects the data reported by the Collectors. As a result, all the older entities will be inactive based on the three-hour retention period. You can still access the historical data by removing the Entity Status filter on the entity page. Ensure that you upgrade the AppDynamics Operator and AppDynamics Collectors using the latest Helm charts to avoid any issues. See Upgrade or Uninstall Kubernetes and App Service Monitoring.

You can use this attribute along with its value (UUID of the kube-system namespace) to filter the entity details of the clusters with the same cluster name. See ClustersNamespaces, WorkloadsPodsPersistent Volume Claims, and Ingresses.

The k8s.cluster.id attribute is required to send MELT data for Kubernetes entities. By default, AppDynamics Helm charts and Collectors attach the k8s.cluster.id attribute to data from AppDynamics sources. To send data from third-party collectors, you must enrich your data with the k8s.cluster.id attribute. The k8s.cluster.id attribute must have a value equal to the UUID of the kube-system namespace. See Install Kubernetes and App Service Monitoring.

23.4.0
Kubernetes Infrastructure and Troubleshooting

The following Kubernetes platform versions are now supported:

  • AWS EKS: 1.25
  • Azure AKS: 1.26
  • AWS ROSA: 4.12.9
  • Azure Openshift: 4.11.34

See Install Kubernetes and App Service Monitoring.

23.4.0

Resolved Issues

KeySummaryVersion

OTEL-464

Kubernetes Collectors failed to deploy with Helm chart 1.9.526 after the schema upgrade from clusterName to clusterId. This issue was caused because ArgoCD does not support the Helm lookup function, which is required by appdynamics-cloud-helm-chart 1.9.520 and 1.9.526. The issue was resolved by providing the new value field global.clusterId in appdynamics-cloud-helm-chart 1.9.532.

Users must provide global.clusterId in any scenario where helm lookup is not supported, such as when using ArgoCD, running helm template, or running helm install --dry-run. See Advanced Settings for the Cisco AppDynamics Distribution of OpenTelemetry Collector.

23.4.1

O2-2008

The cluster ID for the K8s clusters name was not a unique identifier and the same named clusters were reporting to the same entity, causing data issues.This issue has now been corrected and resolved.23.4.0

XTNSBL-3310

The promoted tags were missing for Aurora cluster. This issue has now been corrected and resolved.23.4.0

Known Issues and Limitations

Customer Onboarding and Startup

  • We are making constant updates to the site in order to make the user experience better, so the Observe page may not show the most up to date version. If this happens, we recommend clearing your browser's cache by holding down the Shift key and clicking the browser's reload button (or pressing Ctrl+F5 on a Windows computer or Cmd+Shift+R on a Mac computer). The browser will refresh and load the most recent version of the page.
  • Every user has the ability to configure and view all data. This release does not support more granular access control.
  • Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

  • Cisco Cloud Observability is available in AWS US West, AWS Sydney and AWS Frankfurt regions only. See About Cisco Cloud Observability and SaaS Domains and IP Ranges

  • For the HostProcess containers, the AppDynamics Collectors do not collect metrics from Windows Exporter. This happens because Windows HostProcess container monitoring is not supported.  

Kubernetes Infrastructure and Troubleshooting

  • On Clusters and Namespaces pages, the Persistent Volume Claim (PVC) events are not displayed in Kubernetes Events > View Event Summary. However, these events are listed in the Events detail view.
    To view the PVC events, click Kubernetes Events > Show All Events.
  • Cisco Cloud Observability may contain links to documentation for AppDynamics APM, which is a completely different product. Workaround: Use only the documentation on https://docs.appdynamics.com/appd-cloud
  • The Properties panel shows limited metadata.
  • If the cluster you connect to Cisco Cloud Observability already has AppDynamics APM agents running on it, the metrics you see on the Observe page may be different from the metrics you see on your AppDynamics APM Controller. Comparing metrics on the two platforms may yield inconclusive results. There are multiple reasons for this. Cisco Cloud Observability only ingests metrics from AWS CloudWatch, Azure Monitor, and OpenTelemetry™-compatible collectors, whereas AppDynamics App Server agents, such as Machine Agent, send metrics in a proprietary, non-OpenTelemetry format. In addition, those agents are connected to your Controller UI, not to Cisco Cloud Observability. 

  • Some metrics are not displayed for ingress and service entities. Cisco Cloud Observability only displays metrics if your ingress controller is the Kubernetes NGINX controller. There is no support for displaying metrics from other ingress controllers at this time. In addition, if your ingress controller is configured with wildcard domains, there are no metrics for that ingress. To get metrics in this case you need to configure the ingress controller with --metrics-per-host=false (you will lose labeling by hostname, but still have labeling by ingress). 

Service-oriented Investigations

No known issues.

App Root Cause Analysis using Anomaly Detection

No known issues.

Application Troubleshooting with Logs

  • The Log Collector does not filter or scrub out any sensitive data present in log messages. However, you can configure it to drop (omit) fields you specify. See Log Collector Settings. 
  • On the Logs page, the Log Patterns tab does not appear when you use regular browser mode. Workaround: Use a browser in incognito mode. 

OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.

March Release - 23.3.x 

Enhancements

CategorySummaryVersion
Database Monitoring

This release includes a bug fix and new versions of the following AppDynamics artifacts: 

  • Database Collector Docker image: 23.3.1-627
  • AppDynamics Collectors Helm chart: 1.8.483
23.3.30
Kubernetes Infrastructure and TroubleshootingThe AppDynamics Operator Helm chart is updated with the version 1.8.103.23.2.1
Kubernetes Infrastructure and Troubleshooting

This release includes new versions of the following AppDynamics artifacts:

  • OTel Docker image: 23.3.0-790 (for Linux) and 23.3.0-765 (for Windows)
  • Cluster and Infrastructure Collectors Docker image: 23.3.0-1448
  • AppDynamics operator Docker image: 23.3.0-964
  • Log Collector Docker image: 23.3.0-546
  • Database Collector Docker image: 23.3.0-611
  • AppDynamics Collectors Helm chart: 1.8.476
  • AppDynamics Operator Helm chart:  1.9.120
23.3.23

App Root Cause Analysis using Anomaly Detection

For business transactions, the Anomaly Detection algorithm now detects the daily and weekly seasonality in the metric levels- Average Response Time, Calls per Minute, and Errors per Minute. The algorithm now takes into account the seasonal nature of your business. As a result, you get more accurate alerts. See Anomaly Detection.

23.3.23
Application Troubleshooting with Logs

The Logs page is updated with:

    • Download button
    • Customizable columns in the log message table​
    • A count of events in the specified time range
    • Reduced height histogram to give more screen space to log messages
    • Hide Histogram button

See Troubleshoot with Logs.

23.3.23
Application Troubleshooting with Logs

Log Collector improvements includes the following:

  • You can now deploy the Log Collector on Kubernetes clusters running on Windows containers. 
  • You can now run the helm upgrade command to apply changes to your cluster, instead of helm uninstall and helm install.
23.3.23
Cloud Infrastructure and Troubleshooting 

AppDynamics now supports infrastructure monitoring for Kubernetes Windows container workloads in EKS and AKS distributions. You can now monitor the key performance metrics and health of your Windows container workloads.

See:

23.3.23
Cloud Infrastructure and Troubleshooting You can now monitor the following Azure entities:23.3.23
Cloud Infrastructure and Troubleshooting 

AppDynamics now supports monitoring non-Kubernetes Windows hosts running in AWS, Azure, or your private environment.

For setup and configuration details, see:

23.3.23

Cloud Infrastructure and Troubleshooting 

loglevel is deprecated. The default appdynamics-otel-collector logging exporter setting has changed from loglevel: debug to verbosity: detailed. Continuing to use loglevel to configure the appdynamics-otel-collector logging exporter will cause a crash.

23.3.23


Database Monitoring

Database Collector installation support in the multi-tenant mode. See, Install Database Monitoring on Kubernetes.

23.3.23
Kubernetes and App Service Monitoring

The Kubernetes and App Service Monitoring supports monitoring the Windows nodes in a cluster. However, the Operators will run in the Linux node only. See Install Kubernetes and App Service Monitoring.

The os and env parameters are introduced to specify your configuration based on the operating system. See AppDynamics Collectors Settings.

23.3.23
Service-oriented Investigations

The Spans and Traces list views now has a filter panel where you can filter spans and traces quicker and easier. Also, the Traces detail view has a new Trace Flowmap where you can group by service, span, or entities, where their color correlates with the relevant span in the timeline. See Traces View Details.

23.3.23

Resolved Issues

KeySummaryVersion
ALERT-13354Health Rule violations were not displayed in some environments because the metadata for some health rules is incorrect. The environment issues were corrected so these health rule violations are being displayed. 23.1.7
DSO-588The Database Collector container does not start if you use the Database Collector scratch image.23.3.30

O2-2347

The AppDynamics Operator installation fails because of the generic cluster role resource names used in the Operator collides with other roles in your Kubernetes environment.

23.2.1
XTNSBL-3073The Observe page used to display fewer than 10 ECS services, so if there were more than 10,  they would not be displayed. The problem was that the AWS describe-services (DescribeServicesRequest) was limited to fetching 10 service names. The limit was increased and this issue, as a result, was fixed.23.3.6-2219

ALERT-13293, PRE-9823

The values displayed in the Health Rule violations summary did not correlate with the actual metrics shown in the inline chart. This was because there was a gap in time aggregation logic for metrics in Health Rule services and metrics services. This issue has now been corrected and resolved.

23.3.16
ALERT-12550, PRE-8757The Health Rule violations were not being evaluated for a specific K8s cluster. The issue was corrected and now can be evaluated for the K8s cluster.22.12.5

Known Issues and Limitations

Customer Onboarding and Startup

  • Every user has the ability to configure and view all data. This release does not support more granular access control.
  • Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

  • Cisco Cloud Observability is available in AWS US West, AWS Sydney and AWS Frankfurt regions only. See About Cisco Cloud Observability and SaaS Domains and IP Ranges

  • For the HostProcess containers, the AppDynamics Collectors do not collect metrics from Windows Exporter. This happens because Windows HostProcess container monitoring is not supported. 

     

Kubernetes Infrastructure and Troubleshooting

  • On Clusters and Namespaces pages, the Persistent Volume Claim (PVC) events are not displayed in Kubernetes Events > View Event Summary. However, these events are listed in the Events detail view.
    To view the PVC events, click Kubernetes Events > Show All Events.
  • Cisco Cloud Observability may contain links to documentation for AppDynamics APM, which is a completely different product. Workaround: Use only the documentation on https://docs.appdynamics.com/appd-cloud
  • The list view displays a maximum of 1000 entities of one type.
  • The Properties panel shows limited metadata.
  • If the cluster you connect to Cisco Cloud Observability already has AppDynamics APM agents running on it, the metrics you see on the Observe page may be different from the metrics you see on your AppDynamics APM Controller. Comparing metrics on the two platforms may yield inconclusive results. There are multiple reasons for this. Cisco Cloud Observability only ingests metrics from AWS CloudWatch, Azure Monitor, and OpenTelemetry™-compatible collectors, whereas AppDynamics App Server agents, such as Machine Agent, send metrics in a proprietary, non-OpenTelemetry format. In addition, those agents are connected to your Controller UI, not to Cisco Cloud Observability. 


  • Some metrics are not displayed for ingress and service entities. Cisco Cloud Observability only displays metrics if your ingress controller is the Kubernetes NGINX controller. There is no support for displaying metrics from other ingress controllers at this time. In addition, if your ingress controller is configured with wildcard domains, there are no metrics for that ingress. To get metrics in this case you need to configure the ingress controller with --metrics-per-host=false (you will lose labeling by hostname, but still have labeling by ingress). 

Service-oriented Investigations

No known issues.

App Root Cause Analysis using Anomaly Detection

No known issues.

Application Troubleshooting with Logs

  • The Log Collector does not filter or scrub out any sensitive data present in log messages. However, you can configure it to drop (omit) fields you specify. See Log Collector Settings. 
  • On the Logs page, the Log Patterns tab does not appear when you use regular browser mode. Workaround: Use a browser in incognito mode. 

OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.

February Release - 23.2.x 

Enhancements

CategorySummaryVersion
Kubernetes Infrastructure and TroubleshootingThe AppDynamics Operator Helm chart is updated with the version 1.8.103.23.2.1
Database Monitoring

This release includes minor UI bug fixes and displays the database instance name on the query list page.

23.2.6
Cloud Infrastructure and Troubleshooting You can now monitor the following AWS entities:23.2.23
Application Troubleshooting with Logs

The Logs page is updated with:

  • Outlier detection improvements 
  • Histogram improvements
  • Share button​

See: Troubleshoot with Logs.

23.2.23
Kubernetes Infrastructure and Troubleshooting

This release includes new versions of the following AppDynamics artifacts:

  • appdynamics-collectors Helm chart 1.6.378
23.2.23
Service-oriented Investigations

AppDynamics Cloud now provides you the ability to group using Group View. You can view the count of entities in groups to understand how the entities are distributed, such as health and metrics. See Grouping.

23.2.23
Database Monitoring

Support for Database Collector installation using:

23.2.23
Observability

In HTTP request actions, you can now provide the entity_context_path variable in the predefined variables and add your tenant-base URL to the variable to form a deep link URL. The deep link URL helps you to directly navigate to the entity page in your application when a health rule violates and view the context of the entity at the time of the violation. See Predefined Variables.

23.2.23
Observability

To improve user experience, following enhancements are made:

  • In the Create Health Rule wizard, the Filter by Tags and Attributes field now supports free text values.
  • In the Create Health Rule wizard, the Link Action(s) step now shows the name of the actions that you have already linked.
  • In the Create HTTP Request Action wizard, the default method of Request URL is now changed to POST instead of GET.

See Define Entity Details and Create HTTP Request Action.

23.2.23
App Root Cause Analysis using Anomaly Detection

AppDynamics Cloud now automatically detects anomalies in service endpoints by using the Anomaly Detection algorithm. For the Service Endpoint entity type, you can also configure Anomaly Detection by applying filters using tags and attributes and selecting the sensitivity level. See Configure Anomaly Detection.

23.2.23
App Root Cause Analysis using Anomaly Detection

The Anomaly Detection algorithm now detects the daily and weekly seasonality in the metric levels- Average Response Time, Calls per Minute, and Errors per Minute. The algorithm now takes into account the seasonal nature of your business. As a result, you get more accurate alerts. See Anomaly Detection.

23.2.23

Optimization

AppDynamics now has a service configuration enhancement that includes the Topology Store which has been upgraded on one cluster. Also, there are now new application configuration possibilities for UQL services.23.2.27

Resolved Issues

KeySummaryVersion

PRE-5161

Redirection to Duo SSO Idp is now successful.

23.2.23
O2-2347The AppDynamics Operator installation fails because the generic cluster role resource names used in the Operator collide with other roles in your Kubernetes environment23.2.1

Known Issues and Limitations

Customer Onboarding and Startup

  • Every user has the ability to configure and view all data. This release does not support more granular access control.
  • Cisco Cloud Observability is available in AWS US West, AWS Sydney and AWS Frankfurt regions only. See About Cisco Cloud Observability and SaaS Domains and IP Ranges

Kubernetes Infrastructure and Troubleshooting

  • On Clusters and Namespaces pages, the Persistent Volume Claim (PVC) events are not displayed in Kubernetes Events > View Event Summary. However, these events are listed in the Events detail view.
    To view the PVC events, click Kubernetes Events > Show All Events.
  • Cisco Cloud Observability may contain links to documentation for AppDynamics APM, which is a completely different product. Workaround: Use only the documentation on https://docs.appdynamics.com/appd-cloud
  • The list view displays a maximum of 1000 entities of one type.
  • The Properties panel shows limited metadata.
  • If the cluster you connect to Cisco Cloud Observability already has AppDynamics APM agents running on it, the metrics you see on the Observe page may be different from the metrics you see on your AppDynamics APM Controller. Comparing metrics on the two platforms may yield inconclusive results. There are multiple reasons for this. Cisco Cloud Observability only ingests metrics from AWS CloudWatch, Azure Monitor, and OpenTelemetry™-compatible collectors, whereas AppDynamics App Server agents, such as Machine Agent, send metrics in a proprietary, non-OpenTelemetry format. In addition, those agents are connected to your Controller UI, not to Cisco Cloud Observability. 

  • Some metrics are not displayed for ingress and service entities. Cisco Cloud Observability only displays metrics if your ingress controller is the Kubernetes NGINX controller. There is no support for displaying metrics from other ingress controllers at this time. In addition, if your ingress controller is configured with wildcard domains, there are no metrics for that ingress. To get metrics in this case you need to configure the ingress controller with --metrics-per-host=false (you will lose labeling by hostname, but still have labeling by ingress). 

Service-oriented Investigations

No known issues.

App Root Cause Analysis using Anomaly Detection

No known issues.

Application Troubleshooting with Logs

  • The Log Collector does not filter or scrub out any sensitive data present in log messages. However, you can configure it to drop (omit) fields you specify. See Log Collector Settings. 
  • On the Logs page, the Log Patterns tab does not appear when you use regular browser mode. Workaround: Use a browser in incognito mode. 

OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.

January Release - 23.1.x

Enhancements

CategorySummaryVersion
Cloud Infrastructure and TroubleshootingAppDynamics now supports Host Monitoring for cloud (non-Kubernetes) and non-cloud hosts. See Host Monitoring.
The Host Monitoring entities you can monitor are:
23.1.30
Application Troubleshooting with Logs

Log Collector deployments that use the advanced layout of collectors-values.yaml must rename the configuration parameter monitoring.otlp to monitoring.otlpmetric. See Advanced AppDynamics Log Collector Settings.

23.1.30
Application Troubleshooting with LogsThe Log Collector now provides extra ways to express conditions for log collection. See:
23.1.30
Application Troubleshooting with LogsLog Collector deployments that use the advanced layout of collectors-values.yaml must rename or move certain configuration parameters related to the OTLP codec. See Advanced AppDynamics Log Collector Settings23.1.30
Application Troubleshooting with LogsThe Log Collector is now based on Filebeat v8.2.3, which includes support for harvesting logs from cronjobs and for configuring batch sizes. Batch sizes based on bytes / max_bytes. See AppDynamics Log Collector Settings.23.1.30
Application Troubleshooting with Logs

The Logs page has some updates:

  • There's a new tab, Log Patterns, that groups log messages based on similar structure and signature patterns. See Troubleshoot with Logs.
  • Outliers, previously only visible on the histogram of an entity's detail view, are now visible on the histogram of the Logs page for service instances.
23.1.30
Kubernetes Infrastructure and TroubleshootingAppDynamics provides OpenShift support for self-managed linux configurations. See Software Requirements.23.1.30
Kubernetes Infrastructure and Troubleshooting

This release includes new versions of the following AppDynamics artifacts:

  • OTel Docker image 23.1.0-625
  • appdynamics-cloud-k8s-monitoring Docker image 23.1.0-1408
  • appdynamics-cloud-k8s-operator Docker image 22.10.0-753
  • LCA Docker image 23.1.0-512
  • DB Collector Docker image 23.1.0-435
  • appdynamics-collectors Helm chart 1.6.378
  • appdynamics-operator Helm chart  1.6.89
23.1.30
Kubernetes Infrastructure and Troubleshooting

The following deprecated keys have been removed:

  • appdynamics-cloud-k8s-monitoring.clusterName
  • appdynamics-cloud-k8s-monitoring.clustermonConfig.clusterName
  • appdynamics-cloud-k8s-monitoring.containermonConfig.clusterName
  • appdynamics-cloud-k8s-monitoring.logCollectorConfig.clusterName

You can use the global.clusterName key.

23.1.30
Kubernetes Infrastructure and Troubleshooting

This release introduces:

  • A new entity page for Persistent Volume Claim (PVC) monitoring. See Persistent Volume Claims.
  • The PVC metrics breakdown for each PVC along with existing aggregated metrics at cluster and namespace level.
  • The PVC usage metrics at cluster, namespace  and PVC level. See PVC metrics at Clusters, Namespaces, and Persistent Volume Claims.
23.1.30
Kubernetes Infrastructure and Troubleshooting

Pod Status Events metric is introduced for the Clusters, Namespaces, Workloads, and Pods detail view pages.

This metric displays the status of the pod reported every minute.

23.1.30
Service-oriented InvestigationsAppDynamics Cloud now enables you to observe Span entity data in greater detail. A Span represents a unit of operation or work. Spans are the building blocks of Traces because every span provides specific, additional details about a transaction. See Spans.
23.1.30
Service-oriented InvestigationsYou can now view Traces with new data visualizations that includes a Service Graph and mini-map. See Traces View Details.23.1.30
App Root Cause Analysis using Anomaly Detection

AppDynamics Cloud now automatically detects anomalies in business transaction by using the Anomaly Detection algorithm. For the business transaction entity type, you can also configure Anomaly Detection by applying filters using tags and attributes and selecting the sensitivity level. See Configure Anomaly Detection.

23.1.30
App Root Cause Analysis using Anomaly Detection

In Observe > Business Transactions page, you can now view the suspected causes of an anomaly detected for a particular business transaction. The suspected cause information helps to determine the call paths and root cause of an anomaly, presenting you with the information you need to quickly form and verify a hypothesis. See Determine the Root Cause of an Anomaly.

23.1.30
Database Monitoring
  • Query profiling is now available for SQL server.
  • One-minute granular KPIs are added for SQL server:
    • Average Response Time (ms)
    • Total Elapsed time (secs)    
    • Executions / Minute
  • You can create health rules on database query metrics.

See Database Monitoring.

23.1.30
OptimizationThe efficiency and risk profiles of your Kubernetes workloads now display in the central pane of the AppDynamics Cloud UI and includes the metrics average replicas, cpu utilization, and memory utilization. See Workload Efficiency and Risk Profile.23.1.30
AppDynamics Cloud IntegrationsThis release introduces AppDynamics Cloud Grafana plugin to integrate AppDynamics Cloud with Grafana. You can now view AppDynamics Cloud data on Grafana dashboards. See Integrate Cisco Cloud Observability with Grafana.23.1.30



Resolved Issues

KeySummaryVersion
ALERT-13354Health Rule violations were not displayed in some environments because the metadata for some health rules is incorrect. The environmental issues were corrected so these health rule violations are being displayed. 23.1.7

O2-2068

Pod status metrics are of gauge type. During time aggregation, there is a loss of critical information such as pod failures.

23.1.0

Known Issues and Limitations

Customer Onboarding and Startup

  • Every user has the ability to configure and view all data. This release does not support more granular access control.
  • Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

  • Cisco Cloud Observability is available in AWS US West, AWS Sydney and AWS Frankfurt regions only. See About Cisco Cloud Observability and SaaS Domains and IP Ranges

Kubernetes Infrastructure and Troubleshooting

  • On Clusters and Namespaces pages, the Persistent Volume Claim (PVC) events are not displayed in Kubernetes Events > View Event Summary. However, these events are listed in the Events detail view.
    To view the PVC events, click Kubernetes Events > Show All Events.
  • Cisco Cloud Observability may contain links to documentation for AppDynamics APM, which is a completely different product. Workaround: Use only the documentation on https://docs.appdynamics.com/appd-cloud
  • The list view displays a maximum of 1000 entities of one type.
  • The Properties panel shows limited metadata.
  • If the cluster you connect to Cisco Cloud Observability already has AppDynamics APM agents running on it, the metrics you see on the Observe page may be different from the metrics you see on your AppDynamics APM Controller. Comparing metrics on the two platforms may yield inconclusive results. There are multiple reasons for this. Cisco Cloud Observability only ingests metrics from AWS CloudWatch, Azure Monitor, and OpenTelemetry™-compatible collectors, whereas AppDynamics App Server agents, such as Machine Agent, send metrics in a proprietary, non-OpenTelemetry format. In addition, those agents are connected to your Controller UI, not to Cisco Cloud Observability. 


  • Some metrics are not displayed for ingress and service entities. Cisco Cloud Observability only displays metrics if your ingress controller is the Kubernetes NGINX controller. There is no support for displaying metrics from other ingress controllers at this time. In addition, if your ingress controller is configured with wildcard domains, there are no metrics for that ingress. To get metrics in this case you need to configure the ingress controller with --metrics-per-host=false (you will lose labeling by hostname, but still have labeling by ingress). 

Service-oriented Investigations

No known issues.

App Root Cause Analysis using Anomaly Detection

No known issues.

Application Troubleshooting with Logs

  • The Log Collector does not filter or scrub out any sensitive data present in log messages. However, you can configure it to drop (omit) fields you specify. See Log Collector Settings. 
  • Logs from cronjobs and jobs, which are part of a workload, are not visible on the workload's entity centric page. 

  • On the Logs page, the Log Patterns tab does not appear when you use regular browser mode. Workaround: Use a browser in incognito mode. 

OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.

December Release - 22.12.x 

Enhancements

CategorySummaryVersion

Cloud Infrastructure and Troubleshooting

AppDynamics Cloud uses Container Insights to collect AWS Elastic Container Services (ECS) metrics. AppDynamics Cloud supports the following container services for AWS:

22.12.15

Kubernetes Infrastructure and Troubleshooting

AppDynamics has documented the recommended configurations to filter sensitive data at the Collector level. See Filter Sensitive Data Using the Collector.22.12.15

Service-oriented Investigations


Business Transactions (BTs) are now visible on the Observe page. A Business Transaction is a visual representation of the transactions (request and response) that take place for the services on the application. BTs give a more granular view of services and interactions based on its starting point. It represents the processing path that is followed in response to a user-initiated request–in other words, the logical path of a user through your services. For this release, AppDynamics Cloud BTs are disabled by default. Please work with your field representative to turn them on for your installation. See: 

22.12.15
Service-oriented Investigations

AppDynamics Cloud enables you to observe Traces entity data in greater detail. A trace represents a transaction that traveled through AppDynamics Cloud. A trace is made of one or more Spans, including one for each unit or service that handled that transaction. Without traces, it is challenging to pinpoint the cause of performance issues. For this release, AppD Cloud Trace Exploration is disabled by default. Please work with your field representative to turn them on for your installation See:

22.12.15
App Root Cause Analysis using Anomaly DetectionYou can now configure Anomaly Detection for Services and Service Instances through the user interface and use the machine learning capabilities for the automatic detection of performance issues. While configuring Anomaly Detection, you can select the sensitivity level (High, Medium, or Low) of the Anomaly Detection algorithm and link the HTTP Request Actions that are available in your Cloud Tenant. See Configure Anomaly Detection.22.12.15







Resolved Issues

KeySummaryVersion

INFRAMON-3354

The attribute for ReplicaSet name in the schema has been corrected. You can now filter Kafka Cluster, Topic, and Consumer Group entities by ReplicaSet name on the entity list view and on the Configure page when you create health rules.

22.12.15

Known Issues and Limitations

Customer Onboarding and Startup

  • Every user has the ability to configure and view all data. This release does not support more granular access control.
  • Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

  • Cisco Cloud Observability is available in AWS US West, AWS Sydney and AWS Frankfurt regions only. See About Cisco Cloud Observability and SaaS Domains and IP Ranges

Kubernetes Infrastructure and Troubleshooting

  • Cisco Cloud Observability may contain links to documentation for AppDynamics APM, which is a completely different product. Workaround: Use only the documentation on https://docs.appdynamics.com/appd-cloud
  • The list view displays a maximum of 1000 entities of one type.
  • The Properties panel shows limited metadata.
  • If the cluster you connect to Cisco Cloud Observability already has AppDynamics APM agents running on it, the metrics you see on the Observe page may be different from the metrics you see on your AppDynamics APM Controller. Comparing metrics on the two platforms may yield inconclusive results. There are multiple reasons for this. Cisco Cloud Observability only ingests metrics from AWS CloudWatch, Azure Monitor, and OpenTelemetry™-compatible collectors, whereas AppDynamics App Server agents, such as Machine Agent, send metrics in a proprietary, non-OpenTelemetry format. In addition, those agents are connected to your Controller UI, not to Cisco Cloud Observability. 


  • Some metrics are not displayed for ingress and service entities. Cisco Cloud Observability only displays metrics if your ingress controller is the Kubernetes NGINX controller. There is no support for displaying metrics from other ingress controllers at this time. In addition, if your ingress controller is configured with wildcard domains, there are no metrics for that ingress. To get metrics in this case you need to configure the ingress controller with --metrics-per-host=false (you will lose labeling by hostname, but still have labeling by ingress). 


Service-oriented Investigations

No known issues.

App Root Cause Analysis using Anomaly Detection

No known issues.

Application Troubleshooting with Logs

  • The Log Collector does not filter or scrub out any sensitive data present in log messages. However, you can configure it to drop (omit) fields you specify. See Log Collector Settings.
  • Logs from k8s jobs, which are part of a workload, are not visible on the workload's entity-centric page.


OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.




November Release - 22.11.x 

Enhancements

CategorySummaryVersion
Cloud Infrastructure and Troubleshooting

Schema Change
On November 17, 2022, we pushed a schema change that affects the Amazon RDS monitoring experience. As a result, Amazon RDS data from before the change is now inaccessible, and existing Amazon RDS health rules no longer work. 

Required Actions

Recreate your Amazon RDS health rules. See Create Health Rules to Monitor Entities or Group of Entities.
22.11.17
Cloud Infrastructure and Troubleshooting

Updates to the the list/detail views of entities include:

  • Azure databases are now visible. You can now collect and correlate metrics from Azure SQL and PostgreSQL.
  • Amazon RDS observability includes redesigned modeling of Aurora clusters and instances. 
22.11.17
Kubernetes Infrastructure and Troubleshooting

Service entities now include a logs histogram.

22.11.17

Kubernetes Infrastructure and Troubleshooting

You can now monitor your Redis and Kafka entities via Prometheus-based exporters to Cisco Cloud Observability.

22.11.17
Kubernetes Infrastructure and TroubleshootingYou can now monitor Red Hat® OpenShift® clusters running on AWS or Azure infrastructure and managed offerings such as Red Hat OpenShift Service on AWS (ROSA) and Azure managed OpenShift. For the list of supported versions, see Software Requirements in Install Kubernetes and App Service Monitoring.22.11.17
Kubernetes Infrastructure and Troubleshooting

We've updated the AppDynamics Operator and Cluster Collector:

Required Actions

To get these updates, upgrade the AppDynamics components on your clusters. See Upgrade or Uninstall Kubernetes and App Service Monitoring.
22.11.17

Application Troubleshooting with Logs

Use the Log Parsing Validator to test parsing patterns for your log messages. This tool displays the fields it extracts from a single-line log message, parser type, and pattern you provide. When you're happy with the parsing, you can copy the successful pattern into your Log Collector configuration. This leads to faster onboarding of logs. See Log Parsing Validator.

22.11.17
Application Troubleshooting with Logs

We've updated the Logs page:

  • Breadcrumbs make it easier for you to navigate back to the entity you came from.
  • Group logs by property/metric/tag.
  • View log messages associated with an event when the event ID is included in the log messages.
  • Searching by keyword has a new syntax:  raw = '<keyword>'. Always use single quotes. If you use double quotes or no quotes, auto-complete is unavailable. 
  • The "load" progress of search results now has a visual element: On the histogram, a progress bar moves from right (newest timestamp) to left (oldest timestamp) as the page loads the search results.
  • A new button, Explore All Logs, removes the entity filter so that you see an all logs in the time range you've selected. You can also navigate to this unfiltered view from Explore > Logs.
  • Sort log messages by oldest-first or newest-first.

See Explore Logs.

22.11.17
Application Troubleshooting with Logs

You can now configure the Log Collector with subparsers for Grok log messages in the new (June 2022) format of collector-values.yaml. When you specify a subparser, you can extract more fields from different parts of the log message. See Log Collector Settings.

22.11.17
Application Troubleshooting with Logs

Outliers are temporarily disabled on the log histogram of entity detail pages.

22.11.17

Resolved Issues

KeySummaryVersion
PRE-7986,
TMNT-5477
Tags with `-` character in their name were not applied in filters.22.11.17

Known Issues and Limitations

Customer Onboarding and Startup

  • Every user has the ability to configure and view all data. This release does not support more granular access control.
  • Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

  • Cisco Cloud Observability is available in AWS US West, AWS Sydney and AWS Frankfurt regions only. See About Cisco Cloud Observability and SaaS Domains and IP Ranges

Kubernetes Infrastructure and Troubleshooting

  • Cisco Cloud Observability may contain links to documentation for AppDynamics APM, which is a completely different product. Workaround: Use only the documentation on https://docs.appdynamics.com/appd-cloud
  • The list view displays a maximum of 1000 entities of one type.
  • The Properties panel shows limited metadata.
  • If the cluster you connect to Cisco Cloud Observability already has AppDynamics APM agents running on it, the metrics you see on the Observe page may be different from the metrics you see on your AppDynamics APM Controller. Comparing metrics on the two platforms may yield inconclusive results. There are multiple reasons for this. Cisco Cloud Observability only ingests metrics from AWS CloudWatch, Azure Monitor, and OpenTelemetry™-compatible collectors, whereas AppDynamics App Server agents, such as Machine Agent, send metrics in a proprietary, non-OpenTelemetry format. In addition, those agents are connected to your Controller UI, not to Cisco Cloud Observability. 

  • Some metrics are not displayed for ingress and service entities. Cisco Cloud Observability only displays metrics if your ingress controller is the Kubernetes NGINX controller. There is no support for displaying metrics from other ingress controllers at this time. In addition, if your ingress controller is configured with wildcard domains, there are no metrics for that ingress. To get metrics in this case you need to configure the ingress controller with --metrics-per-host=false (you will lose labeling by hostname, but still have labeling by ingress). 

Service-oriented Investigations

No known issues.

App Root Cause Analysis using Anomaly Detection

No known issues.

Application Troubleshooting with Logs

  • The Log Collector does not filter or scrub out any sensitive data present in log messages. However, you can configure it to drop (omit) fields you specify. See Log Collector Settings.
  • Logs from cronjobs and jobs, which are part of a workload, are not visible on the workload's entity centric page. 


OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.



October Release - 22.10.x 

Enhancements

CategorySummaryVersion
Kubernetes Infrastructure and Troubleshooting

This release includes new versions of the following AppDynamics artifacts:

  • appdynamics-cloud-k8s-monitoring Docker image 22.10.0-1366
  • appdynamics-cloud-operator Docker image 22.10.0-753
  • appdynamics-collectors Helm chart  1.4.201
  • appdynamics-operator Helm chart  1.4.42

These updates include a fix described in Resolved Issues. To upgrade, see Upgrade or Uninstall Kubernetes and App Service Monitoring.

22.10.20
Kubernetes Infrastructure and Troubleshooting

In collectors-values.yaml, we've added a new parameter for specifying the name of your cluster, global.clusterName

For example,

global:
  clusterName: sampleCluster
YML

The value of this new parameter is propagated to all AppDynamics collectors (clustermon, containermon, logCollector). 

Required Actions

  1. Add global.clusterName to your collectors-values.yaml and set its value to the name of your cluster as it appears on the Observe page. 

  2. Remove the following older parameters:
    • appdynamics-cloud-k8s-monitoring.clusterName
    • appdynamics-cloud-k8s-monitoring.clustermonConfig.clusterName
    • appdynamics-cloud-k8s-monitoring.containermonConfig.clusterName
    • appdynamics-cloud-k8s-monitoring.logCollectorConfig.clusterName
22.10.20
Kubernetes Infrastructure and TroubleshootingOn the detail view of all Kubernetes entities except clusters, the Properties panel now includes Kubernetes annotations. 22.10.20
Service-oriented Investigations

Cisco Cloud Observability automatically correlates your AKS-based services with entities from your AKS infrastructure. To enable this feature, configure all OpenTelemetry SDKs, AppDynamics collectors, and hybrid agents to provide the same values for container.id and k8s.pod.uid

22.10.20
Service-oriented Investigations

There are some differences on the detail views of some entities as we transition to a modern UI framework:

  • Service instances now include a logs histogram.
22.10.20
Application Troubleshooting with Logs

Use these new features on the Logs page:

  • You can now drag-select a time range directly on the logs histogram.
  • View log messages associated with a trace or span when those trace or span IDs are included log messages.
  • The new filter component utilized on the Logs page now allows you to remove all filters to provide an "unfiltered" view of log messages.
  • Searching by keyword now requires you to use the new query syntax. To search for all log messages that include Example, instead of entering keyword = 'Example' , enter type = 'Example'.

See Explore Logs.

22.10.20
Application Troubleshooting with Logs

We've added support for subparsers for Grok log messages. When you specify a subparser, you can extract more fields from different parts of the log message. This feature is available for collector-values.yaml in legacy YAML layouts only. See the subparsers setting in Log Collector Settings - Advanced YAML Layout.

22.10.20
Metrics

We've enhanced the AppDynamics Metrics Model with:

  • Cumulative metrics–we convert them into Delta before storing them.
  • New metric categories: RATE and RATE_PER_SEC

See Cisco AppDynamics Metrics Model.

22.10.20
Optimization

Cisco Cloud Observability enables you to review the efficiency and risk profiles of your Kubernetes workloads. It identifies and quantifies performance, reliability risks, and cost reduction opportunities for actionable adjustments that you can make to your workloads. See Workload Efficiency and Risk Profile.

22.10.20
API

Cisco Cloud Observability introduces the Action API (Beta) that enables you to create, configure, and manage various actions. An action is a predefined, reusable, automated response to an event such as a health violation or an anomaly. See Cisco Cloud Observability Action API.

22.10.20



Resolved Issues

KeySummaryVersion
PRE-8510, 
METRICSVC-8868

Cisco Cloud Observability stopped aggregating metrics for subtypes of different entity types briefly. Metric aggregation is now working again. For example, when we aggregate metrics to workload entities, if the actual entity is a subtype of workload, like deployment or statefulset, we will aggregate metrics to the sub type entities as well

22.10.07-0020
PRE-8629,
LCA-338

The Log Collector applied the default configuration for log collection to all containers, which caused it to harvest logs from unintended containers.

Required Actions

To apply the fix for this issue, you must upgrade the appdynamics-collectors Helm chart 1.4.201. See Upgrade or Uninstall Kubernetes and App Service Monitoring.

22.10.20

Known Issues and Limitations

Customer Onboarding and Startup

  • Every user has the ability to configure and view all data. This release does not support more granular access control.
  • Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP


  • Cisco Cloud Observability is available in AWS Local Zone us-west-2-pdx-1a only. This means that any data you send to Cisco Cloud Observability, including any personal data, is stored in the United States in this AWS Local Zone, and should only be provided to AppDynamics if you have the appropriate rights and consent for such provision. See About Cisco Cloud Observability and SaaS Domains and IP Ranges.

Kubernetes Infrastructure and Troubleshooting

  • Cisco Cloud Observability may contain links to documentation for AppDynamics APM, which is a completely different product. Workaround: Use only the documentation on https://docs.appdynamics.com/appd-cloud
  • The list view displays a maximum of 1000 entities of one type.
  • The Properties panel shows limited metadata.
  • If the cluster you connect to Cisco Cloud Observability already has AppDynamics APM agents running on it, the metrics you see on the Observe page may be different from the metrics you see on your AppDynamics APM Controller. Comparing metrics on the two platforms may yield inconclusive results. There are multiple reasons for this. Cisco Cloud Observability only ingests metrics from AWS CloudWatch, Azure Monitor, and OpenTelemetry™-compatible collectors, whereas AppDynamics App Server agents, such as Machine Agent, send metrics in a proprietary, non-OpenTelemetry format. In addition, those agents are connected to your Controller UI, not to Cisco Cloud Observability. 


  • Some metrics are not displayed for ingress and service entities. Cisco Cloud Observability only displays metrics if your ingress controller is the Kubernetes NGINX controller. There is no support for displaying metrics from other ingress controllers at this time. In addition, if your ingress controller is configured with wildcard domains, there are no metrics for that ingress. To get metrics in this case you need to configure the ingress controller with --metrics-per-host=false (you will lose labeling by hostname, but still have labeling by ingress). 

Service-oriented Investigations

No known issues.

App Root Cause Analysis using Anomaly Detection

No known issues.

Application Troubleshooting with Logs

  • The Log Collector does not filter or scrub out any sensitive data present in log messages. However, you can configure it to drop (omit) fields you specify. See Log Collector Settings.
  • Logs from cronjobs and jobs, which are part of a workload, are not visible on the workload's entity centric page. 


OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.

September Release - 22.9.x

Enhancements

CategorySummaryVersion
Cloud Infrastructure and TroubleshootingView additional Azure entities like disks and standard load balancers. See Azure Disk Storage and Azure Standard Load Balancer.22.9.23
Kubernetes Infrastructure and Troubleshooting

Use these new predefined health rules to monitor your Kubernetes entities:

  • K8s Pod Restart Count
  • K8s Node Resource Memory, PID, Disk Pressure

The health rules are only available on tenants provisioned from v22.9.

In addition, for CPU and memory usage, your health rules can use these new rollup paths for unmanaged ReplicaSets and unmanaged jobs:

  • AWS EC2 to Kubernetes node to Kubernetes cluster
  • Azure VM to Kubernetes node to Kubernetes cluster

See Kubernetes Entities.

22.9.23
Kubernetes Infrastructure and Troubleshooting

Cluster Collector now supports monitoring the standalone or unmanaged pods.

22.9.23
Kubernetes Infrastructure and Troubleshooting

When a container in a pod is killed because of OOM (Out of Memory), the OOMKilled event is generated on the pod entity. See Pod Event Reasons.

22.9.23
Kubernetes Infrastructure and Troubleshooting

This release includes new versions of the following AppDynamics collectors:

  • appdynamics-cloud-k8s-monitoring: Docker image 22.9.0-1340
  • appdynamics-cloud-operator: Docker image 22.9.0-738
  • appdynamics-collectors Helm chart  1.3.169
  • appdynamics-operator Helm chart  1.3.169
22.9.23
Service-oriented Investigations

Notice some differences on the detail views of some AWS entities as we transition to a modern UI framework:

22.9.23
Service-oriented InvestigationsThe AppDynamics Metrics Model now supports cumulative metrics instead of delta metrics. In addition, it supports rate and rate/sec metrics.22.9.23

Application Troubleshooting with Logs

Notice some differences related to logs as we transition to a modern UI framework:

  • You can now view logs associated with a trace or span.
  • The logs histogram is available on the detail views of service instances.
  • The Filter bar, logs list, and Properties panel have an updated look.

See Explore Logs.

22.9.23
Application Troubleshooting with Logs

We've added support for advanced configurations in the Log Collector. You can now add a fallback or default condition for harvesting logs from many containers on your cluster. This allows for faster setup. To get this update, you must migrate logCollectorConfig key in collectors-values.yaml to the September 2022 layout. See 2024-03-27_06-19-25_Migrate logCollectorConfig from August to September 2022 Format.

22.9.23
App Root Cause Analysis using Anomaly Detection
AppDynamics Cloud now performs the root cause analysis of an anomaly within the instances of a service. The root cause analysis data helps you to quickly troubleshoot the anomaly. See Determine the Root Cause of an Anomaly.
22.9.23
App Root Cause Analysis using Anomaly DetectionThe predefined variable names that are used in an HTTP request action are updated to support the MELT events. If you are using these predefined variables in the payload of HTTP request action, you must update them. Otherwise, the HTTP request action does not work. See Predefined Variables.22.9.23
Troubleshooting with RCAIn your development and pre-production environments, you can now test anomaly detection by setting the modelSensitivity  level as TEST in the API. See AppDynamics Cloud Anomaly Detection Configuration API.22.9.23
APIAppDynamics Cloud introduces the Health Rules API (Beta) to create, update, and manage one or more health rules for monitored entities. See AppDynamics Cloud Health Rules API.

Resolved Issues

KeySummaryVersion
O2-1972

On the Kubernetes event details page, the data was not displayed within the event details table. The event details table now displays the data.

22.9.23
PRE-8007, 
DASHBASE-1817
Retrieving logs for a cluster, namespace, workload, pod, or host entity now works on historic data (in other words, data from before an internal schema change on August 29) as well as recently indexed data.22.9.23
TMNT-5477The tags with "-" character in the tag key name are not applied in Filters. The problem has been corrected so that filters on custom AWS tags whose value contains no-alpha numeric characters can be handled gracefully in AppD Cloud.22.9.8

ALERT-12241

The action triggered API payload caused an issue for creating health rules. The required API payload was corrected and the issue is now resolved. 

22.9.22

Known Issues and Limitations

Customer Onboarding and Startup

  • Every user has the ability to configure and view all data. This release does not support more granular access control.
  • Service Provider-initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through Access Management > User Management > View User > Authenticated By > My IdP

  • Cisco Cloud Observability is available in AWS Local Zone us-west-2-pdx-1a only. This means that any data you send to Cisco Cloud Observability, including any personal data, is stored in the United States in this AWS Local Zone, and should only be provided to AppDynamics if you have the appropriate rights and consent for such provision. See About Cisco Cloud Observability and SaaS Domains and IP Ranges.

Kubernetes Infrastructure and Troubleshooting

  • Cisco Cloud Observability may contain links to documentation for AppDynamics APM, which is a completely different product. Workaround: Use only the documentation on https://docs.appdynamics.com/appd-cloud
  • The list view displays a maximum of 1000 entities of one type.
  • The Properties panel shows limited metadata.
  • If the cluster you connect to Cisco Cloud Observability already has AppDynamics APM agents running on it, the metrics you see on the Observe page may be different from the metrics you see on your AppDynamics APM Controller. Comparing metrics on the two platforms may yield inconclusive results. There are multiple reasons for this. Cisco Cloud Observability only ingests metrics from AWS CloudWatch, Azure Monitor, and OpenTelemetry-compatible collectors, whereas AppDynamics App Server agents, such as Machine Agent, send metrics in a proprietary, non-OpenTelemetry format. In addition, those agents are connected to your Controller UI, not to Cisco Cloud Observability. 

  • Some metrics are not displayed for ingress and service entities. Cisco Cloud Observability only displays metrics if your ingress controller is the Kubernetes NGINX controller. There is no support for displaying metrics from other ingress controllers at this time. In addition, if your ingress controller is configured with wildcard domains, there are no metrics for that ingress. To get metrics in this case you need to configure the ingress controller with --metrics-per-host=false (you will lose labeling by hostname, but still have labeling by ingress). 

Service-oriented Investigations

No known issues.

App Root Cause Analysis using Anomaly Detection

No known issues.

Application Troubleshooting with Logs

  • The Log Collector does not filter or scrub out any sensitive data present in log messages. However, you can configure it to drop (omit) fields you specify. See Log Collector Settings.
  • Logs from cronjobs and jobs, which are part of a workload, are not visible on the workload's entity centric page. 

OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.



August Release - 22.8.x

Enhancements

CategorySummaryVersion
Kubernetes Infrastructure and TroubleshootingOnboard AWS cloud connections at scale using a CloudFormation template to automate the creation of the AWS IAM policy and role. See Connect to AWS Using Role Delegation.22.8.29-1700
Kubernetes Infrastructure and Troubleshooting

This release includes new versions of the following AppDynamics collectors:

  • AppDynamics Distribution for OpenTelemetry™ Collector: Docker image 22.8.0-391
  • appdynamics-cloud-k8s-monitoring: Docker image 22.8.0-1305
  • appdynamics-cloud-operator: Docker image 22.8.0-727
  • AppDynamics Log Collector: Docker image 22.8.0-304
  • appdynamics-collectors Helm chart  1.2.122
  • appdynamics-operator Helm chart  1.2.122

The collectors contain bug fixes described in Resolved Issues. To update your collectors, see Upgrade or Uninstall Kubernetes and App Service Monitoring.

22.8.29-1700
API

The predefined variable names that you can use in HTTP requests have been renamed. For example, eventId has been renamed to event_id.

Required Actions

If you are using these predefined variables in the payload of an HTTP request, you must update them. Otherwise, the HTTP request will fail. See Predefined Variables.
22.8.29-1700
API

Cisco Cloud Observability Anomaly Detection Configuration API (Beta) helps you to create Anomaly Detection configurations for specific entities and entity types. You can also set a sensitivity level such as high, medium, low, and test for the Anomaly Detection configurations as per your business needs. See AppDynamics Cloud Anomaly Detection Configuration API.

22.8.29-1700
Service-oriented Investigations

You'll start to see some differences in the user interface as we transition the user interface to a modern UI framework. For this hybrid release, the Application Performance Monitoring pages are updated:

  • The filter/search box on service and service instance pages now includes more keyboard support making it quicker for you to create and edit applied filters. See Filters. To view Service and Service Instances pages, you may have to hard refresh the browser. See Services, Services Detail, Service Instances, and Service Instances Detail.
  • Single-clicking a flow map entity displays metadata in the Properties panel.
  • The Properties panel contains data for both services and remote services (backends).
  • Select multiple service instances for one detail view of all of them.
22.8.29-1700
Service-oriented InvestigationsTo see historical topology and health, use a custom time range. See Understand the Observe UI.22.8.29-1700
App Root Cause Analysis using Anomaly Detection

Get an aggregated view of the health of the entities you monitor. Previously, Cisco Cloud Observability determined the health of an entity only by the health rules associated with it. Now, Cisco Cloud Observability determines the health of an entity by aggregating the health rules associated with it and the anomaly detection algorithm. See Entity Health Monitoring.

22.8.29-1700
Application Troubleshooting with LogsConfigure the Log Collector to apply multiple parsers on logs. See Log Collector Settings - Advanced YAML Layout.22.8.29-1700
Application Troubleshooting with LogsConfigure the Log Collector to send its metrics to your own metrics backend. This gives you visibility into its performance. 22.8.29-1700
Application Troubleshooting with LogsWe've improved the format of the Log Collector's settings in collectors-values.yaml. These improvements require you to migrate your YAML file to a new format. See Log Collector Settings.22.8.29-1700
Application Troubleshooting with Logs

The logCollector section of your AppDynamics collectors' configuration file, collectors-values.yaml, has separated the parameters related to metrics export from those related to logging configuration. 

Required Actions

If you have enabled Filebeat logging on the Log Collector, replace deprecated configuration parameters with new ones in  collectors-values.yaml. See Log Collector Settings.
22.8.29-1700





Resolved Issues

KeySummaryVersion
ALPS-4972Error "unable to load data" for My Profile > Current Tenants. Corrections were made to fix API issues for the Mini Profile.22.8.30
INFRAMON-3001The Infrastructure Manager starts its own instance of the node exporter on the default port (9100). You can modify this port setting through its configuration file, collectors-values.yaml: Set servermonConfig:exporterPort. See AppDynamics Collectors Settings.22.8.29-1700
INFRAMON-3020,
O2-1931,
LCA-242

If your clusters use StatefulSets for pod setup, those pods and their containers are now correctly associated on Relationship maps and Properties panels.

Symptoms of incorrect or missing associations were:

  • When you redeployed a StatefulSet, the Relationships map incorrectly associated the newer pods and containers with older pods and containers.
  • When you had StatefulSets with the same pod and namespace names on different clusters, the Relationships map incorrectly associated pods and containers from both clusters. 
  • On the Logs page, the Properties panel contained conflicting values for the attributes cluster and k8s.cluster.name. 


Required Actions

To obtain this fix you must:

  1. Specify the name of your cluster in a new parameter, appdynamics-cloud-k8s-monitoring.clusterName, in collectors-values.yaml. The cluster name must match the name displayed in Cisco Cloud Observability. See AppDynamics Collectors Settings
    For example,

    appdynamics-cloud-k8s-monitoring:
      clusterName: "sampleClusterName"
    YML

    The value of this new parameter is propagated to all AppDynamics collectors (clustermon, containermon, logCollector). This parameter replaces the following older parameters, which you can now delete: 

    appdynamics-cloud-k8s-monitoring:
     ...
      clustermonConfig:
        clusterName: "sampleClusterName"
        ...
      containermonConfig:
        clusterName: "sampleClusterName"
        ...
      logCollectorConfig:
        container:
          clusterName: "sampleClusterName"
          ...
    YML
  2. Update all AppDynamics operators and collectors running on your clusters. See Upgrade or Uninstall Kubernetes and App Service Monitoring
  3. If you're using a legacy layout of the logCollectorConfig key in collectors-values.yaml, you must add this script to logCollectorConfig.filebeatYaml.processors.  Replace <cluster-name> with the name of your cluster, and retain the colon suffix. See Log Collector Settings - Advanced YAML Layout

    processors:
      ...
      - script:
          lang: javascript
          source: >
            function process(event) {
              var podUID = event.Get("k8s.pod.uid");
              if (podUID) {
                event.Put("internal.container.encapsulating_object_id", "<cluster-name>:" + podUID);
              }
              return event;
            }
    YML


Backward compatibility:

Older versions of the AppDynamics collectors will work, but will still exhibit the symptoms described here if your clusters use StatefulSets for pod setup.  Specifically, if you update the AppDynamics collectors on some of your clusters but not on others, and the clusters running old AppDynamics collectors use StatefulSets with the same pod and namespace names as those on clusters running new AppDynamics collectors, your Relationships maps and Properties panels may exhibit the symptoms described here.

22.8.29-1700
INFRAMON-3022 O2-1910

AppDynamics collectors now send the Azure resource ID (attribute/tag azure.resource.id) for correct association between Kubernetes nodes and Azure VMs.   

Symptoms of incorrect or missing associations were:

  • When you selected an Azure host, its detail view didn't display data for Kubernetes metrics (disk, memory, PID pressure).
  • On a container's or pod's detail view, the Properties panel didn't display any Azure tags.


Required Actions

To obtain this fix you must update all AppDynamics operators and collectors running on your clusters. See Upgrade or Uninstall Kubernetes and App Service Monitoring

22.8.29-1700
O2-1933After completing the Kubernetes and App Service Monitoring installation, the data is not displayed on the UI and the logs show that the Collectors are not reporting data. This happens if your ingress controller uses ingress rules with resourceBackends or if the rule path is not specified (can happen in the case of master ingresses).22.8.29-1700
PRE-7758For service entities, the Topology and Relationships maps now show connections to remote services like backend databases.22.8.15-1600
PRE-7879Metrics are now visible for AWS Application Load Balancers and Network Load Balancers.22.8.29-1700

Known Issues and Limitations

Customer Onboarding and Startup

  • Every user has the ability to configure and view all data. This release does not support more granular access control.
  • Service Provider-initiated SSO is not working (does not redirect to your identity provider), so you can't auto-federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through Access Management > User Management >View User > Authenticated By > My IdP

  • Cisco Cloud Observability is available in AWS Local Zone us-west-2-pdx-1a only. This means that any data you send to Cisco Cloud Observability, including any personal data, is stored in the United States in this AWS Local Zone, and should only be provided to AppDynamics if you have the appropriate rights and consent for such provision. See About Cisco Cloud Observability and SaaS Domains and IP Ranges.

Kubernetes Infrastructure and Troubleshooting

  • Cisco Cloud Observability may contain links to documentation for AppDynamics APM, which is a completely different product. Workaround: Use only the documentation on https://docs.appdynamics.com/appd-cloud
  • The list view displays a maximum of 1000 entities of one type.
  • The Properties panel shows limited metadata.
  • If the cluster you connect to Cisco Cloud Observability already has AppDynamics APM agents running on it, the metrics you see on the Observe page may be different from the metrics you see on your AppDynamics APM Controller. Comparing metrics on the two platforms may yield inconclusive results. There are multiple reasons for this. Cisco Cloud Observability only ingests metrics from AWS CloudWatch, Azure Monitor, and OpenTelemetry-compatible collectors, whereas AppDynamics App Server agents, such as Machine Agent, send metrics in a proprietary, non-OpenTelemetry format. In addition, those agents are connected to your Controller UI, not to Cisco Cloud Observability. 

  • Some metrics are not displayed for ingress and service entities. Cisco Cloud Observability only displays metrics if your ingress controller is the Kubernetes NGINX controller. There is no support for displaying metrics from other ingress controllers at this time. In addition, if your ingress controller is configured with wildcard domains, there are no metrics for that ingress. To get metrics in this case you need to configure the ingress controller with --metrics-per-host=false (you will lose labeling by hostname, but still have labeling by ingress). 

Service-oriented Investigations

No known issues.

App Root Cause Analysis using Anomaly Detection

No known issues.

Application Troubleshooting with Logs

  • The Log Collector does not filter or scrub out any sensitive data present in log messages. However, you can configure it to drop (omit) fields you specify. See Log Collector Settings.
  • Logs from cronjobs and jobs, which are part of a workload, are not visible on the workload's entity centric page. 

OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.


July Release - 22.7.x

Enhancements

CategorySummary
ObservabilityMonitor Kubernetes® event details on the new Events page. Filter the content of this page by event reason, severity, or tags. See View Event Details.
ObservabilityGroup by tags: On the Observe page, you can group data by the following global tags: 
  • env
  • owner
  • cluster
  • namespace 
  • service 
  • version
  • aws.account.id
  • azure.location
  • aws.region
  • azure.resource_group
  • azure.subscription.id

Filter by tags: On the Observe page, you can filter by entity type, entity status, tag, or health. On list views, you can filter by attribute, entity status, tag, or health.

Observability

View an entity's violating metrics, along with indicators for metric thresholds and baselines: On the Violating Metric chart, you can now view the baseline and threshold values of a violating metric for an entity. Baseline and threshold indicators help you to quickly examine the details of a violation. This chart displays different information depending on what you've defined in your health rules:

  • For health rules where a single metric threshold is defined, the chart displays the threshold value.
  • For health rules where baselines are used, the chart displays the baseline, mean value, mean value plus the standard deviation range, and mean value minus the standard deviation range.

See Health Violation Timeline.

Observability

The redesigned Health Rollup Path field provides improved usability and benefits such as:

  • Easy selection of health rollup paths from the dropdown list if available for a selected entity.
  • Better search capability to find the desired health rollup paths for a selected entity.

See Define Entity Details.

ObservabilityUse the equal ("=") operator in your health rule condition to assign a static literal value to a metric. The condition becomes true when the specified value for the metric is met. See Define Health Rule Evaluation Conditions.
Observability

Cisco Cloud Observability now detects anomalies within service instances. The anomaly data helps pinpoint the exact metric associated with the service instance that is violating and also provides other details that help you take corrective actions. See Monitor Anomalies.

Observability

Cisco Cloud Observability has updated the anomaly detection algorithm that calculates baselines. This algorithm helps to reduce false alerts and improves issue detection within application services.

OnboardingUse tags and expressions to limit the cloud service instances you monitor. See Configure the Cloud Connection, Connect Your AWS IAM User Account to AppDynamics, and Add Custom Configuration for Azure Connection
Onboarding

This release includes new versions of the following Cisco Cloud Observability artifacts:

  • appdynamics-cloud-k8s-monitoring: Docker image 22.7.0-1283
  • appdynamics-cloud-operator: Docker image 22.7.0-706
  • appdynamics-collectors Helm chart  1.1.83
  • appdynamics-operator Helm chart  1.1.83



Resolved Issues

KeySummary
PRE-7212If your Kubernetes environment does not allow you to pull images from any outside public repository, you can now specify the location of the kube-rbac-proxy image in your collectors-values.yaml file. 
PRE-7223,
PRE-7221

The Logs page now displays multiline log messages related to namespace, workload, and pod entities.

Known Issues and Limitations

Accounts

  • Every user has the ability to configure and view all data. This release does not support more granular access control.
  • Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto federate users into your account. Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP

Observability

  • Cisco Cloud Observability may contain links to documentation for AppDynamics APM, which is a completely different product. Workaround: Use only the documentation on https://docs.appdynamics.com/appd-cloud
  • The list view displays a maximum of 1000 entities of one type.
  • The Properties panel shows limited metadata.
  • Logs from cronjobs and jobs, which are part of a workload, are not visible on the workload's entity centric page. 

  • If the cluster you connect to Cisco Cloud Observability already has AppDynamics APM agents running on it, the metrics you see on the Observe page may be different from the metrics you see on your AppDynamics APM Controller. Comparing metrics on the two platforms may yield inconclusive results. There are multiple reasons for this. Cisco Cloud Observability only ingests metrics from AWS CloudWatch, Azure Monitor, and OpenTelemetry-compatible collectors, whereas AppDynamics App Server agents, such as Machine Agent, send metrics in a proprietary, non-OpenTelemetry format. In addition, those agents are connected to your Controller UI, not to Cisco Cloud Observability. 

  • Some metrics are not displayed for ingress and service entities. Cisco Cloud Observability only displays metrics if your ingress controller is the Kubernetes NGINX controller. There is no support for displaying metrics from other ingress controllers at this time. In addition, if your ingress controller is configured with wildcard domains, there are no metrics for that ingress. To get metrics in this case you need to configure the ingress controller with --metrics-per-host=false (you will lose labeling by hostname, but still have labeling by ingress). 

Onboarding

  • Cisco Cloud Observability is available in AWS Local Zone us-west-2-pdx-1a only. This means that any data you send to Cisco Cloud Observability, including any personal data, is stored in the United States in this AWS Local Zone, and should only be provided to AppDynamics if you have the appropriate rights and consent for such provision. See About Cisco Cloud Observability and SaaS Domains and IP Ranges.

  • The Log Collector does not filter or scrub out any sensitive data present in log messages. However, you can configure it to drop (omit) fields you specify. See Log Collector Settings.

OpenTelemetry™ and Kubernetes® (as applicable) are trademarks of The Linux Foundation®.


June Release - 22.6.x

Features

For an overview of Cisco Cloud Observability, see About Cisco Cloud Observability and Observe UI Overview.

Account Management Portal

  • The Account Management portal provides easy access to university subscriptions, professional services, community, and support contracts associated with your account. 
  • All users of a license have the ability to configure and view all data. This release does not support more granular access control.
  • Expand a license to see your tenant's licensing tier, which is based on your tenant's current token count. The licensing tier determines your tenant's rate limits. 
  • The Projection tab provides current and future token consumption estimates, which help you to verify that you have enough tokens to cover future usage. It also provides an estimation of how many additional tokens you may need, in the event of a shortfall, to ensure continuous monitoring for the current license unit anniversary period. See Manage Licenses.
  • Review your license usage limit on a per-tenant or per-entitlement basis. Overage Pool lets you view license overage utilization by month, anniversary, or a custom time window. 

APIs

  • The Application Principal Management API enables administrators to create, list, update, and delete application principals. See Agent Principal Management API.
  • The Cloudmon API enables administrators to configure filters on each cloud connection to restrict the data that Cisco Cloud Observability ingests. Filter by cloud service (such as EC2 or EBS), metric (such as CPU utilization and IOPs), or tags. This reduces the costs of data collection by collecting only the data you need. See AppDynamics Cloud Connection API.

Onboarding

  • When you create a cloud connection, Cisco Cloud Observability collects all available data from supported cloud services by default. On the Configure page,  configure filters on each cloud connection to restrict the data that Cisco Cloud Observability ingests to reduce data collection costs. This page supports filtering by cloud service and region. You can also pause data collection. This allows your CloudOps team to create the connection ahead of time.

  • Cisco Cloud Observability provides a single collector that supports full-stack monitoring and correlation of applications and Kubernetes infrastructure. To generate authentication credentials necessary for deploying Kubernetes Full-Stack Monitoring on your cluster, navigate to Configure > Kubernetes and APM. See Kubernetes and App Service Monitoring.
  • When you connect Cisco Cloud Observability to Azure Monitor, you can observe your Azure VMs. See Azure Virtual Machines.
  • Send cloud tags to Cisco Cloud Observability so that you can use these tags to group entities on the Observe page. To send tags from AWS, add the permission "tag:GetResources". See Attach Permissions Policies Using JSON

Feedback

  • Give feedback directly within Cisco Cloud Observability with the floating Feedback button. You can leave general comments or issues, attach files or screenshots, and rate usability and product capabilities.

Observability and Monitoring

  • Differentiate between active entities (those that are sending metrics, events, logs, or health status) and inactive entities to help you troubleshoot faster. 
  • Select from Standard time ranges or create a Custom time range to broaden or narrow your view. Standard time ranges are predefined options that span time ranges from minutes to months. Custom enables you to set parameters for a different time range.
  • Use predefined shortcuts to group entities by health, tags, or attributes.  

    Not all the shortcuts are available in deployments at this time. 

  • Predefined health rules help you to monitor the health status of entities:
    • Edit predefined rules to meet your requirements. See Health Rules.
    • Define new health rules to monitor the performance of entities or a group of entities in your environment. You define conditions within your health rule that represent the expected behavior of the entities. Any variation in the expected behavior is reported as a health violation. You can initiate a remediation action to mitigate the performance issue. See Entity Health Monitoring.
    • Health rules are only applicable after you configure a cloud connection and deploy infrastructure collectors on your Kubernetes cluster.
  • Anomaly detection helps automate uses AI capabilities to enable root cause analysis (RCA) to show you the suspected causes for every anomaly in the application. Anomalies may be related to latency, errors, and throughput. Anomaly Detection RCA reduces the Mean Time to Resolution (MTTR) by pointing to the source of the problem. You can confirm or negate a suspected cause with a brief look at available data. You can drill down into the data to quickly determine the root cause of any anomaly. See Determine the Root Cause of an Anomaly.
  • The logs histogram displays log counts and highlights outliers–log messages that signal an abrupt increase or decrease in the message count, an abrupt change in severity, or a change in trend. Use outliers to speed up log analysis. 
  • On the Logs page:
    • Filter by attributes or keywords. 
    • Use the search box on the Fields panel to search a selected message for the name or value of a field or attribute within it.
    • View and apply your 20 most recent filters.
    • See Explore Logs.

Known Issues and Limitations

KeySummary

Every user has the ability to configure and view all data. This release does not support more granular access control.


Cisco Cloud Observability may contain links to documentation for AppDynamics APM, which is a completely different product.

Workaround: Use only the documentation that was distributed with this release, rather than anything else online.


The Properties panel shows limited metadata.


On the Observe page, the list view displays a maximum of 1000 entities of one type.


The Log Collector does not filter or scrub out any sensitive data present in log messages. However, you can configure it to drop (omit) fields you specify. See Log Collector Settings.

Cisco Cloud Observability may contain links to documentation for AppDynamics APM, which is a completely different product. Workaround: Use only the documentation on https://docs.appdynamics.com/appd-cloud

SSEG-10309

Cisco Cloud Observability is available in AWS Local Zone us-west-2-pdx-1a only. This means that any data you send to Cisco Cloud Observability, including any personal data, is stored in the United
States in this AWS Local Zone, and should only be provided to AppDynamics if you have the appropriate rights and consents for such provision.

DASHBASE-1258

Logs from cronjobs and jobs, which are part of a workload, are not visible on the workload's entity centric page.

PRE-4100

If the cluster you connect to Cisco Cloud Observability already has AppDynamics APM agents running on it, the metrics you see on your Cisco Cloud Observability Observe page may be different from the metrics you see on your AppDynamics APM Controller. Comparing metrics on the two platforms may yield inconclusive results. There are multiple reasons for this. Cisco Cloud Observability only ingests metrics from AWS CloudWatch, Azure Monitor, and OpenTelemetry-compatible collectors, whereas AppDynamics App Server agents, such as Machine Agent, send metrics in a proprietary, non-OpenTelemetry format. In addition, those agents are connected to your Controller UI, not to Cisco Cloud Observability.

PRE-5100, O2-1674

Some metrics are not displayed for ingress and service entities. Cisco Cloud Observability only displays metrics if your ingress controller is the Kubernetes NGINX controller. There is no support for displaying metrics from other ingress controllers at this time. In addition, if your ingress controller is configured with wildcard domains, there are no metrics for that ingress. To get metrics in this case you need to configure the ingress controller with --metrics-per-host=false (you will lose labeling by hostname, but still have labeling by ingress).

PRE-5161

Service-provider initiated SSO is not working (does not redirect to your identity provider), so you can't auto federate users into your account.

Workaround: On the Account Management Portal, assign your identity provider to each user through User Management > View User > Authenticated By > My IdP.