On this page:

Related pages:

This page describes how to install and administer the Events Service as a single node or cluster on Microsoft Windows systems. The instructions on this page are organized by deployment scenario, as follows:

More than one instance of the same service type is not supported on Windows.

You do not need to specify the installation or data directory for the Events Service installation. If you do, use a different one from the platform directory.

About Master Nodes

As described in the installation steps in the sections below, the exact configuration values you'll use depend on whether you are deploying a single node installation, a three-node cluster, or the fourth or higher node of a multi-node cluster.

Primarily the differences have to do with the role of the master node in the cluster. A master node in the Events Service deployment is a node which—in addition to acting as a storage node—manages the state of the data across the cluster. In a single node deployment, there's not much for the master to do. But in a multi-node cluster, the master manages the state across the entire cluster, including the state of the replica. The first node that starts up acts as the master. If the master node becomes unavailable for any reason, the remaining nodes need to elect a new master.  

The configuration for each node needs to specify: 

The differences by node order are:

Cluster node numberMaster-enabled?Minimum master nodes
Only node in a one-node installationtrue1
First, second, and third nodes of a three-node clustertrue2
Fourth (or higher) node of a clusterfalse2

Deploying an Events Service Node (1 Node)

In the Enterprise Console GUI, the Express Install option can be used to automatically install an Events Service on the same host as the Controller. You can also use the steps below to deploy manually.

To install the Events Service on a single node: 

  1. Unzip the Events Service distribution archive to a directory on the target host. This creates the events-service directory with the Events Service artifacts.  
    You will need to configure a few setting in the Events Service download. But first, configure the Controller to connect to the Events Service and get the connection key, as described next.  

  2. Configure the connection to the Events Service in the Controller, and get the Controller key for the Events Service configuration as follows:

    1. With the Controller running, open the Administration Console as the root user. 
    2. In the Controller Settings page, search for appdynamics.on.premise.event.service.url.
    3. Replace the default value to the internal hostname for the Events Service machine and default Events Service listen port, 9080. If you are putting a load balancer in front of the Events Service (required for a cluster), this will be the VIP for the Events Service as exposed at the load balancer. In this case, it is likely you will need to return to this step after you finish deploying the node and configuring the load balancer. 
    4. Search for an additional setting, one you will need to enable the connection from the Events Service to the Controller, appdynamics.on.premise.event.service.key
    5. Copy the value of the property to your clipboard. You will need configure this in the Events Service properties file next. 

  3. Configure the connection from the Events Service to the Controller:

    1. In a terminal, navigate to the  events-service directory:

      cd events-service
    2. Open the conf\events-service-api-store.properties file for editing

    3. Find the following property and replace controller-key with the copied key: 

      ad.accountmanager.key.controller=controller-key
  4. Scroll to the end of the events-service-api-store.properties file and verify the minimum and maximum heap settings for the two Events Service processes. Note that the setting values use g for gigabyte (GB) and m for megabyte (MB). 

  5. Save and close the events-service-api-store.properties file. 

  6. Install the Events Service as a Windows service:  

    bin\events-service.exe service-install -p conf\events-service-api-store.properties --auto-start 

    The optional auto-start flag causes the Events Service to be installed as an automatically started service. If you do not include the flag, the Events Service is installed as a manually started service. An additional option, ‑‑log-verbose, increases the verbosity of installation and operation logging, which is useful for troubleshooting.  

  7. Enter the following command to find the service name for the Events Service:

    bin\events-service.exe service-list
  8. Pass the service name returned by service-list command as the -s parameter argument in the following command: 

    bin\events-service.exe service-start -s "<Name from service-list>" 

    Be sure to enclose the name in double quotes. 

  9. Check the health of the new node.  

    bin\events-service.exe check-health -hp localhost:9081

    For the port, pass the administration port for the Events Service, 9081 by default. Verify that "Healthy" appears as the service status, indicating that the process is operating normally:

    [appduser@controller-one events-service]$ bin/events-service.exe check-health -hp 192.168.33.22:9081
    [2015-12-09T18:30:45,342-08:00] HV000001: Hibernate Validator 5.0.2.Final
    [2015-12-09T18:30:45,956-08:00] Individual statuses below:
    [2015-12-09T18:30:45,956-08:00] [192.168.33.22:9081] status is [200 OK]
    [2015-12-09T18:30:45,956-08:00] Overall status Healthy
    ...
  10. Configure the connections from the Analytics Agent, EUM Server, or Database Monitoring agents to the Events Service, as described in Connect to the Events Service

Deploying an Events Service Cluster (3 Nodes)

The following steps describe how to deploy an Events Service cluster made up of three nodes, the minimum size of the Events Service cluster. These steps apply whether you are performing a new installation of a cluster or expanding a single node deployment into a three-node cluster. For information on expanding beyond a three node cluster, see Adding Nodes to a Cluster

Note that all services on Windows machines must be installed on the Enterprise Console host, since the Enterprise Console does not support remote operations on Windows. Therefore, you cannot use the Enterprise Console GUI to deploy an Events Service cluster. 

Before starting, review the topology notes in Events Service Deployment and make sure that all machines in the cluster meet the system requirements.

When ready, perform these steps on each of the three nodes.

  1. Follow the steps for configuring a single node cluster in the 1-node installation above. Additionally, configure the following settings in the  conf\events-service-api-store.properties  file:

    1. Change the value of the ad.es.node.minimum_master_nodes property to 2:

      ad.es.node.minimum_master_nodes=2

      The setting specifies the minimum number of master-eligible instances that must be available in order to elect a new master. Since an Events Service cluster has three master nodes, this value should be two for a cluster.

    2. Set the value of ad.es.event.index.shards to the number of nodes, in this case three:

      ad.es.event.index.shards=3

      You do not need to change this value if it is already higher than the number of nodes.

    3. Set the replication factor to 1 by changing the ad.es.event.index.replicas and ad.es.metadata.replicas properties, as follows:

      ad.es.event.index.replicas=1
      ad.es.event.index.hotLifespanDays=10
      
      ad.es.metadata.replicas=1
    4. For the unicast hosts property, add the hostname or IP address, along with the port 9300, for each node in the cluster:

      ad.es.node.unicast.hosts=node1.example.com:9300,node2.example.com:9300,node3.example.com:9300
    5. Change the publish host to the IP address or hostname of this machine. For example:

      ad.es.node.network.publish.host=node2.example.com
    6. Configure heap space for the Events Service and ElasticSearch processes, as follows:

      1. To set the Events Service process heap size to 1 GB, for example, use the following properties:

        ad.jvm.options.name=events-service.vmoptions
        ad.jvm.heap.min=1g
        ad.jvm.heap.max=1g

        For the setting value, g indicates gigabyte (GB), and m indicates megabyte (MB). 

      2. For the ElasticSearch process, the heap size should be set to half the size of the available RAM on the system, up to a maximum of 31 GB. To set the ElasticSearch process heap size to 8 GB, for example, set the properties as follows:

        ad.es.jvm.options.name=events-service.vmoptions
        ad.es.jvm.heap.min=8g
        ad.es.jvm.heap.max=8g

        For the setting value, g indicates gigabyte (GB), and m indicates megabyte (MB). 

    7. Save and close the file.
  2. Install the Events Service as a Windows service:  

    bin\events-service.exe service-install -p conf\events-service-api-store.properties  --auto-start

    The optional auto-start flag causes the Events Service to be installed as an automatically started service. If you do not include the flag, the Events Service is installed as a manually started service. An additional option, ‑‑log-verbose, increases the verbosity of installation and operation logging, which is useful for troubleshooting.

  3. Enter the following command to find the service name for the Events Service:

    bin\events-service.exe service-list
  4. Pass the service name returned by the service-list command as the -s parameter argument in the following command: 

    bin\events-service.exe service-start -s "<Name from service-list>" 

    Be sure to enclose the name in double quotes. 

  5. Check the health of the new node using the following command. At least two nodes must be running before you run the command. 

    bin\events-service.exe check-health -hp localhost:9081

    For the port, pass the administration port for the Events Service, 9081 by default. Verify that "Healthy" appears as the service status, indicating that the process is operating normally:

    [appduser@controller-one events-service]$ bin/events-service.exe check-health -hp 192.168.33.22:9081
    [2015-12-09T18:30:45,342-08:00] HV000001: Hibernate Validator 5.0.2.Final
    [2015-12-09T18:30:45,956-08:00] Individual statuses below:
    [2015-12-09T18:30:45,956-08:00] [192.168.33.22:9081] status is [200 OK]
    [2015-12-09T18:30:45,956-08:00] Overall status Healthy
    ...
  6. Configure a load balancer to distribute traffic to the Events Service cluster, as described in Load Balance Events Service Traffic

  7. Connect the Controller and other clients—Analytics Agent, EUM Server, or Database Monitoring agents—to the Events Service, as described in Connect to the Events Service.  

Expanding a Cluster (4+ Nodes)

The Events Service cluster is horizontally scalable. You can add nodes to an existing cluster without affecting or having to restart the existing nodes. 

Note that all services on Windows machines must be installed on the Enterprise Console host, since the Enterprise Console does not support remote operations on Windows. Therefore, you cannot use the Enterprise Console GUI to expand an Events Service cluster. 

Before starting, prepare the new cluster machine. Verify system requirements and prepare the environment as described above. 

For each node beyond the original three master nodes, download and configure the nodes as previously described. The configuration steps for any nodes added to the cluster after the initial three master nodes are as follows:

  1. For each cluster nodes beyond the initial three master nodes, open the conf\events-service-api-store.properties for editing and make these configuration changes:

    1. Set the ad.es.node.master value to false: 

      ad.es.node.master=false
    2. Set the ad.es.node.minimum_master_nodes value to 2. 

      ad.es.node.minimum_master_nodes=2
    3. Set the value of ad.es.event.index.shards to the number of nodes in the cluster. You do not need to change this value if it is already higher than the number of nodes.

      ad.es.event.index.shards=<number_of_nodes>

       You do not need to change this value if it is already higher than the number of nodes.

    4. For the unicast hosts property, add the hostnames or IP addresses of all nodes in the cluster, including the node you are adding. For each node specify the ports on which the nodes communicate, 9300-9400. For example:

      ad.es.node.unicast.hosts=node1.example.com[9300-9400],node2.example.com[9300-9400],node3.example.com[9300-9400],node4.example.com[9300-9400]

      You do not need to reconfigure the unicast hosts settings for existing cluster members, as the new node can join the cluster dynamically. 

    5. Change the publish host to the IP address or hostname of this machine. For example:

      ad.es.node.network.publish.host=node4.example.com
    6. Configure heap space for the Events Service and ElasticSearch processes, as follows:

      1. To set the Events Service process heap size to 1 GB, for example, use the following properties:

        ad.jvm.options.name=events-service.vmoptions
        ad.jvm.heap.min=1g
        ad.jvm.heap.max=1g

        For the setting value, g indicates gigabyte (GB), and m indicates megabyte (MB). 

      2. For the ElasticSearch process, the heap size should be set to half the size of the available RAM on the system, up to a maximum of 31 GB. To set the ElasticSearch process heap size to 8 GB, set the properties as follows:

        ad.es.jvm.options.name=events-service.vmoptions
        ad.es.jvm.heap.min=8g
        ad.es.jvm.heap.max=8g

        For the setting value, g indicates gigabyte (GB), and m indicates megabyte (MB). 

    7. Save and close the file.  

  2. Install the Events Service as a Windows service:  

    bin\events-service.exe service-install -p conf\events-service-api-store.properties --auto-start 

    The optional auto-start flag causes the Events Service to be installed as an automatically started service. If you do not include the flag, the Events Service is installed as a manually started service. An additional option, ‑‑log-verbose, increases the verbosity of installation and operation logging, which is useful for troubleshooting.

  3. Enter the following command to find the service name for the Events Service:

    bin\events-service.exe service-list
  4. Pass the service name returned by the service-list command as the -s parameter argument in the following command: 

    bin\events-service.exe service-start -s "<Name from service-list>"  
  5. Check the health of the new node:

    bin\events-service.exe check-health -hp localhost:9081

    Note:  At least two nodes must be running before you run the command.

    For the port, pass the administration port for the Events Service, 9081 by default. Verify that "Healthy" appears as the service status, indicating that the process is operating normally:

    [appduser@controller-one events-service]$ bin/events-service.bin check-health -hp 192.168.33.22:9081
    [2015-12-09T18:30:45,342-08:00] HV000001: Hibernate Validator 5.0.2.Final
    [2015-12-09T18:30:45,956-08:00] Individual statuses below:
    [2015-12-09T18:30:45,956-08:00] [192.168.33.22:9081] status is [200 OK]
    [2015-12-09T18:30:45,956-08:00] Overall status Healthy
    ...
  6. Modify your load balancer rules to include the new cluster node. For more information, see Load Balance Events Service Traffic.

Starting and Stopping the Events Service

At installation, the Events Service is installed as a service and is left running upon completion of the installation. You can stop and stop it as a service or as a foreground process, as described here or by using the GUI.

Starting and Stopping as a Foreground Process

To start the Events Service as a foreground process, however, use the following command: 

bin\events-service.exe start -p conf\events-service-api-store.properties 

To stop the Events Service as a foreground process, use this command: 

bin\events-service.exe stop 

Stopping and Starting as a Windows Service

You can stop and start the Events Service as a Windows service from the Services Manager. You can also stop and start it using the events-service.exe tool, as here: 

  1. Enter the following command to find the service name for the Events Service:

    bin\events-service.exe service-list
  2. Pass the service name returned by the service-list command as the -s parameter argument to the following command. Enclose the service name in double quotes.  

    bin\events-service.exe service-start -s "<Name from service-list>" 

To stop the service, run this command:

bin\events-service.exe service-stop -s "<Name from service-list>"

Removing a Node

To remove a node that is not enabled for operation as a master node from the cluster, simply stop the Events Services on the node or remove the machine it runs on from the network. 

Note the following guidelines: 

After you remove a node, be sure to adjust your load balancer rules to remove the old cluster member. See Load Balance Events Service Traffic for more information. 

If you are not using a load balancer with a cluster deployment, keep in mind that the connection settings for the first master node that reports to the Controller at installation time are written to the Controller setting that identifies the Events Service to the Controller. If you remove a master node in that case, check whether the removed master node is node identified as the Events Service destination URL in the Controller connection settings; adjust the setting if so. See Connect to the Events Service for more information.

To reconfigure an existing node to enable operation as a master node, or add a new node with the master option enabled:

ad.es.node.master=true

If reconfiguring a node, restart the node after changing the configuration.