You can selectively control the number of metrics reported by individual Machine Agents (including Server Visibility Agents). Rather than have all Agents report all the metrics all the time, you can configure individual Agents to collect metrics.

• Key Performance Indicator metrics only (KPI mode)
• KPI and Diagnostic metrics (Diagnostic mode)
• All available metrics (Advanced Diagnostic mode)

This provides the flexibility to report KPI metrics on most machines, and then increase the metric level on specific servers for deeper visibility to diagnose problems. You can increase scalability on the Controller and conserve metric bandwidth on the network with no sacrifice in visibility.

Every Basic and Server Visibility metric has a default Dynamic Monitoring Mode (DMM) class: KPI, Diagnostic, and Advanced Diagnostic. See Hardware Resources Metrics.

Important Notes

• You can disable DMM on individual Agents. When DMM is disabled on an Agent, that Agent will report all metrics regardless of whether DMM is enabled or disabled on the Controller. Disabling DMM on an Agent is recommended only for mission-critical servers and other machines for which you are sure you want to collect all metrics. See Machine Agent Configuration Properties.
• If you switch the monitoring mode in the Controller from a more-inclusive to a less-inclusive mode, the Metric Browser will show values for the newly-excluded metrics for one hour after the switch. For example, if you switch from Diagnostic to KPI mode, for any Diagnostic metric, the Metric Browser will report a steady line (at 0 or the last-reported value) for one hour after the switch; then the line will disappear. This is standard behavior in the Metric Browser for an Agent when it stops reporting a specific metric.
• Each Machine Agent has a set of configurable settings that specify the volumes, networks, and processes to report and to ignore. For example, you can define a process allowlist (report on matching processes) and blocklist (ignore matching processes). An Agent will not report metrics for items excluded by its local settings, regardless of the monitoring mode. See Machine Agent Settings for Server Visibility.
• If you have any custom health rules based on Diagnostic or Advanced Diagnostic metrics, DMM might cause these rules to generate "false-positive" alerts. (Standard, non-custom health rules are not affected.) If you have any health rules similar to this,  edit the rule to use a KPI metric instead of the Diagnostic or Advanced Diagnostic metric.  

Initial Setup

DMM is enabled by default. To configure DMM:

1. Log in to the Controller administration console using the root user password. See 2023-10-19_05-04-05_Access the Administration Console.

   http://<controller host>:<port>/controller/admin.jsp

2. Set these options:
   1. sim.machines.dmm.defaultMode = KPI (Sets the default mode to KPI on all Machine Agents) 
   2. sim.machines.dmm.dmmAllowed = true (Enables Dynamic Monitoring Mode on the Controller)
3. For servers that require additional visibility, increase the monitoring mode to Diagnostic or Advanced Diagnostic. To change the monitoring mode of one or more agents:
   
   1. Log in to the Controller UI.
   2. Click Servers.

   3. In the Servers table, check the box next to one or more servers whose monitoring mode you want to change.

   4. Right-click the server and select Set DMM Mode. A dialog box appears.
   5. Select the desired monitoring mode.
   6. Click Save.

Workflow Example

After you complete the initial setup, you can set the Dynamic Monitoring Mode on individual Machine Agents as required. The following is a workflow example:

• The DevOps team for a large enterprise monitors its IT infrastructure using Machine Agents (1,000-plus Agents monitoring servers in hundreds of locations). All Agents are initially set to KPI mode. 
• One Agent reports a lot of disk read/write operations (KPI metric) on critical-server-A.
• Set the Agent DMM on critical-server-A to Diagnostic.
• Monitor the amount of data read and written for the entire disk, and for each partition (Diagnostic metrics), on critical-server-A.
• If the Diagnostic metrics do not indicate the source of the problem, and further investigation is needed, set the agent DMM to Advanced Diagnostic.
• Monitor average queue times and read/write times for each partition (Advanced Diagnostic metrics) on critical-server-A.
• When advanced diagnostics are no longer required on critical-server-A, set the agent DMM back to KPI.