A remediation script is run on the machine that you specify in the remediation script configuration. You can run the script from the machine affected by the violation that triggered the action or from a central management server. It is not necessary for an app agent to be running on the machine on which the script executes; just a Machine Agent.

Guidelines for Remediation Scripts

By default, the script is a shell script in /bin/sh invoked with the -ex option, unless the script has a header, in which case the interpreter in the header is used. For example, if the script header is #!/bin/perl, the Perl interpreter is invoked.

A process exit code of zero indicates that the script execution succeeded. A non-zero exit code indicates that it failed.

The script should be written as generically as possible to allow it to run on any of the nodes for which is it invoked.  exports the following environment variables to the script runtime to provide context regarding the environment and the event that triggered the action.

Environment Variable

Cardinality (1 or N)

Notes

APP_ID

1

Identity of the application.

APP_NAME

1

Name of the application.

EVENT_TIME

1

Timestamp of the event.

EVENT_ID

1

Identity of the event.

EVENT_TYPE

1

type of event, such as: ERROR, APPLICATION_ERROR, APPLICATION_INFO, STALL, BT_SLA_VIOLATION, DEADLOCK, MEMORY_LEAK, MEMORY_LEAK_DIAGNOSTICS, LOW_HEAP_MEMORY, ALERT, CUSTOM, APP_SERVER_RESTART, BT_SLOW, SYSTEM_LOG, INFO_INSTRUMENTATION_VISIBILITY, AGENT_EVENT, INFO_BT_SNAPSHOT, AGENT_STATUS, SERIES_SLOW, SERIES_ERROR, ACTIVITY_TRACE, OBJECT_CONTENT_SUMMARY, DIAGNOSTIC_SESSION, HIGH_END_TO_END_LATENCY, APPLICATION_CONFIG_CHANGE, APPLICATION_DEPLOYMENT, AGENT_DIAGNOSTICS, MEMORY, LICENSE

ENV_STARTUP_ARGS

1

Process args

ENV_SYSTEM_PROPERTIES

1

JVM System Props, when Java

AFFECTED_ENTITY

1

Affected Entity that triggered the event

SEVERITY

1

Severity level of the event. Possible values are:

  • ERROR- Indicates that the event is of the severity level "error".
  • INFO- Indicates that the event is of the severity level  "information".
  • WARN- Indicates that the event is of the severity level "warning".

Remediation scripts must be stored in a sub-directory of the machine agent installation. The sub-directory must be named local-scripts. The following paths are all valid.

${machine.agent.home}/local-scripts/runMe.sh
${machine.agent.home}/local-scripts/johns_scripts/runMe.sh
${machine.agent.home}/local-scripts/ops/johns_scripts/runMe.sh

Troubleshooting Remediation Scripts

To troubleshoot your remediation script, look for the process in the Machine Agent log. The log location path is: <machine_agent_home>/logs/machine-agent.log

This snippet from the Machine Agent log shows both error and success messages from running a local script named script.sh.

Machine Agent Log