DRAFT SPACE - APPDYNAMICS INTERNAL ONLY

Kubernetes Visibility extends its support to OpenShift Version 3. With this support, you can collect performance data from OpenShift clusters by using the Standalone Machine Agent. You install the agent as a DaemonSet on each host in the OpenShift cluster. The DaemonSet monitors each host on the OpenShift cluster and the corresponding containers that have an AppDynamics App Agent running. The process is similar to that for monitoring Kubernetes clusters. 

The topic provides instructions on how to set up the Standalone Machine Agent running as a DaemonSet on an OpenShift cluster. The process includes the following:

  • Creating the machine-agent project
  • Creating a service account for running the Standalone Machine Agent
  • Building a Docker image of the Standalone Machine Agent 
  • Deploying the Standalone Machine Agent

Before You Start

  • This functionality requires a Server Visibility license.
  • AppDynamics recommends that you use Docker CE/EE v17.03 or Docker Engine v1.13 with this product. Some data might be missing if you are using previous versions of Docker.
  • Make sure you have an OpenShift user account with cluster-admin role, such as system:admin. This user is responsible for configuring the Machine Agent.
  • Install the OpenShift command line tool, oc.

Create Project and Service Account

In OpenShift terminology, a project is a mechanism to isolate a group of users and their resources. An administrator can provide individual users or groups with permissions to create projects or manage specific projects. To isolate the Standalone Machine Agent from other projects, you can create a project for it.

To do so, create the machine-agent project: 

oc new-project machine-agent

Create a service account, ma, with necessary privileges for the Machine Agent to obtain metrics:

  1. Check whether the current project is machine-agent:

    oc status
  2. Create a service account.

    oc create serviceaccount ma

    Service accounts provide a secure way to control OpenShift API access without sharing a regular user’s credentials.

  3. Assign the privileged security context constraints (SCC) to the service account.
    SCC determines the permissions and abilities of pods. See Security Permissions for the Service Account for the detailed list of permissions for the service account.

    oc adm policy add-scc-to-user privileged -z ma
  4. Add cluster-reader to the service account.

    oc adm policy add-cluster-role-to-user cluster-reader -z ma 

Security Permissions for the Service Account

Service account requires the following security permissions to provide maximum isolation to the project under which the Machine Agent is running:

PermissionDescription
The cluster-reader role

This cluster-reader role allows the Machine Agent to read tags from the OpenShift cluster.
An example:

oc adm policy add-cluster-role-to-user cluster-reader -z ma

The privileged SCC

The privileged SCC allows the Machine Agent to be run as the root user.

For example:

oc adm policy add-scc-to-user privileged -z ma

Run as privileged container

Running as the privileged container allows the Machine Agent to perform operations such as:
- Reading files from /etc 
- Reading process information from /proc

This privilege is configured in the DaemonSet YAML file as follows:

securityContext:
          privileged: true         

Creating Docker Image of the Machine Agent

Copy the following files to a directory on a machine that can build the Docker image:

  • Machine Agent Bundle - 64-bit Linux (zip)
    Download this bundle and rename it to machine-agent.zip.

  • Dockerfile
    See the sample Docker file below. 

  • start-appdynamics script
    See the sample script below.  

Build the image by using your desired method and make it available in your image registry.

Deploy Machine Agent as DaemonSet

  1. Modify the sample DaemonSet to point to your Controller. The major sections are highlighted below:
    1. Select the worker node:

      nodeSelector:
              node-role.kubernetes.io/compute: "true"
    2. Change the following Controller information in the sample DaemonSet:

       containers:
            - env:
              - name: APPDYNAMICS_CONTROLLER_HOST_NAME
                value: "<controller-host-name>"
              - name: APPDYNAMICS_CONTROLLER_PORT
                value: "<controller-port>"
              - name: APPDYNAMICS_CONTROLLER_SSL_ENABLED
                value: "true"
              - name: APPDYNAMICS_AGENT_ACCOUNT_ACCESS_KEY
                value: "<account-access-key>"
              - name: APPDYNAMICS_AGENT_ACCOUNT_NAME
                value: "<your-account>" 
              - name: APPDYNAMICS_SIM_ENABLED
                value: "true" 
              - name: APPDYNAMICS_DOCKER_ENABLED
                value: "true"   
    3. Specify the Docker image: 

       image: "<your image>"
    4. Enable the Machine Agent to run as the privileged user:

      securityContext:
                privileged: true
    5. If you have a different service account name, replace it with the existing one:

      serviceAccount: ma
            serviceAccountName: ma
  2. Create DaemonSet below:
    1. Check whether the current project is machine-agent:

      oc status
    2. Create the machine-agent DaemonSet:

      oc create -f machine-agent-daemonset.yaml

      The Machine Agent appears in the UI. 

Check Status

If the Machine agent does not appear in the UI, check the status as follows:

  1. Check the machine-agent DaemonSet.

    oc get ds
  2. Check the machine-agent pod. 

    oc get pod

    The pod name is machine-agent-daemonset-<XXXX>.

  3. Check logs whether the last line is "Started AppDynamics Machine Agent Successfully".

    oc get logs -f <machine-agent-daemonset-<XXXX>>

Sample DaemonSet

Modify the sample DaemonSet to suit your deployment scenario.

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: machine-agent-daemonset 
  namespace: machine-agent 
spec:
  selector:
    matchLabels:
      name: appdynamics-machine-agent
  template:
    metadata:
      labels:
        name: appdynamics-machine-agent
    spec:
      nodeSelector:
        node-role.kubernetes.io/compute: "true"
      containers:
      - env:
        - name: APPDYNAMICS_CONTROLLER_HOST_NAME
          value: "<your-hostname>"
        - name: APPDYNAMICS_CONTROLLER_PORT
          value: "<your-port>"
        - name: APPDYNAMICS_CONTROLLER_SSL_ENABLED
          value: "yes"
        - name: APPDYNAMICS_AGENT_ACCOUNT_ACCESS_KEY
          value: "<your-access-key>"
        - name: APPDYNAMICS_AGENT_ACCOUNT_NAME
          value: "<your-account>" 
        - name: APPDYNAMICS_SIM_ENABLED
          value: "true" 
        - name: APPDYNAMICS_DOCKER_ENABLED
          value: "true" 
        image: <your-image>
        name: machine-agent 
        securityContext:
          privileged: true
        volumeMounts:
        - mountPath: /hostroot
          name: hostroot
          readOnly: true
        - mountPath: /var/run/docker.sock
          name: docker-sock
      restartPolicy: Always
      imagePullPolicy: Always
      serviceAccount: ma
      serviceAccountName: ma
      volumes:
      - name: hostroot
        hostPath:
          path: /
      - name: docker-sock
        hostPath:
          path: /var/run/docker.sock

Sample Dockerfile

You can either use this example Dockerfile or the one given in OpenShift Visibility Manifests.

# Sample Dockerfile for the AppDynamics Standalone Machine Agent
# This is provided for illustration purposes only

FROM ubuntu:16.04
# Install required packages
RUN apt-get update && \
    apt-get install -y unzip && \
    apt-get clean
# Install AppDynamics Machine Agent
ENV MACHINE_AGENT_HOME /opt/appdynamics/machine-agent/
ADD machine-agent.zip /tmp/machine-agent.zip
RUN mkdir -p ${MACHINE_AGENT_HOME} 
# Include start script to configure and start MA at runtime
ADD start-appdynamics ${MACHINE_AGENT_HOME}
RUN chmod 774 ${MACHINE_AGENT_HOME}/start-appdynamics
# change files and directories to be owned by root so MA container user in root 
group can access
RUN chgrp -R 0 /opt && \
    chmod -R g=u /opt
RUN chgrp -R 0 /tmp/machine-agent.zip && \
    chmod g=u /tmp/machine-agent.zip 
# Changing directory to MACHINE AGENT HOME
WORKDIR ${MACHINE_AGENT_HOME}
# Configure and Run AppDynamics Machine Agent
CMD "./start-appdynamics"

Sample start-appdynamics Script

#!/bin/bash
unzip -oq /tmp/machine-agent.zip -d ${MACHINE_AGENT_HOME} && \
    rm /tmp/machine-agent.zip
# Start Machine Agent
./bin/machine-agent -j jre/


Security Considerations

You can deploy the Machine Agent with tighter security. Determine the level of security for the project and leverage the following permissions to secure the Machine Agent.

Run the Machine Agent Without cluster-reader Role

Without the cluster-reader role, the Machine Agent cannot read information, such as Pod and ReplicaSet, from the OpenShift cluster.  However, it can collect other metrics except for the tags for the app server agent container.

Run the Machine Agent Without Privileged Container Mode

To turn off this privilege, remove the section below from the DaemonSet YAML file.

securityContext:
          privileged: true  

Without the Privileged Container mode, the Machine Agent cannot read files, such as/hostroot/etc/passwd and /hostroot/proc/<pid>/etc, and therefore it cannot collect metrics such as:

  • Processes
  • Network

Turning off this privilege has no effect on collecting the app agent container metrics.

  • No labels