Remediation Actions

A remediation action runs a local script in a node. The script executes on the machine from which it was invoked or on the node specified by the remediation action configuration. You can use this type of action to automate your runbook procedures. You can optionally configure the remediation action to require human approval before the script is started. See Actions Requiring Approval.

Remediation actions are not available for servers.
You can create and run a remediation action for a health rule with application, tier, or node as an affected entity. Ensure that you select the same entities when you define the Object Scope for the associated policy.
Select an Application and then create Actions to view the Remediation Action option.

Prerequisites for Local Script Actions

The Machine Agent must be installed running on the host on which the script executes. To see a list of installed Machine Agents for your application, click View machines with machine-agent installed in the bottom left corner of the remediation script configuration window.
To be able to run remediation scripts, the Machine Agent must be connected to an on-premises Controller or to a SaaS Controller via SSL. Remediation script execution is disabled if the Machine Agent connects to a SaaS Controller on an unsecured (non-SSL) HTTP connection.
The Machine Agent OS user must have full permissions to the script file and the log files generated by the script and/or its associated child processes.
The script must be placed in <agent install directory>\local-scripts.
The script must be available on the host on which it executes.
Processes spawned from the scripts must be daemon processes.

Remediation Scripts

A remediation script is run on the machines that you specify in the remediation script configuration. You can run the script from the machine affected by the violation that triggered the action or from a central management server. It is not necessary for an app agent to be running on the machine on which the script executes; just a Machine Agent.

Remediation Example

The remediation action, named increasePool, executes a local script named runbook.sh, which increases the size of the connection pool on the JVM.

Create Remediation Script Action

A policy named ConnectionPoolPolicy triggers this action when the Resource Pool Limit Event starts:

Edit Policy

Create a Local Script (Remediation) Action

To create a remediation action:

Access the Create Action pane. See Create and Modify Actions in Actions.
Select Run a script or executable on problematic Nodes in the Create Action pane and click OK.
After entering a name for the action, in the field that terminates the Relative path to script entry, enter the rest of the path to the executable script.
Remediation scripts must be stored in a sub-directory of the machine agent installation. The sub-directory must be named local-scripts. These paths are all valid:
```
${machine.agent.directory}/local-scripts/runMe.sh
${machine.agent.directory}/local-scripts/johns_scripts/runMe.sh
${machine.agent.directory}/local-scripts/ops/johns_scripts/runMe.sh
```
Click the + to enter the absolute paths of any log files that the script writes to that you want to be included in the script output.
Enter the timeout period for the script process in minutes.
To mandate an approval before the script action can be started, select the Require approval before executing this Action check box and enter the email address of the individual or group that is authorized to approve the action. See Actions Requiring Approval for more information.

Specify the Nodes on Which the Action Will Run

When you bind the action to a policy, you specify the nodes on which the script should execute. You can configure the number of nodes or the percentage of nodes or you can configure a specific node. This flexibility allows you to configure scripts to run from a central management server, not just the node on which the violation occurred.

In the Configure Action pane of the policy actions to execute, either:

Select Execute Action on Affected Nodes and the percentage of the nodes or the number of nodes on which to run the script.
To designate the specific node on which to run the script, select Execute Action on Specified Node, and select the node on which the script should run from the popup node browser. You can either save the configuration or change it to designate a different node.

Review the Local Script Output

In the Events list, locate the row for the event that triggered the action for which you want to see the results.
In the Actions column of the selected row, click the remediation script icon.
In the script results list, select the script output that you want and click Download Local Script Result.

When a remediation action is triggered by a backend discovery event, if the backend is not resolved quickly the policy will not start the local script.