With the standard deployment option, Splunk AppDynamics On-Premises Virtual Appliance installs infrastructure and Splunk AppDynamics Services in your Kubernetes cluster.

Prepare to Install Splunk AppDynamics Services

Complete the following steps to prepare the environment:

1. Log in to one of the node console using the appduser credentials.

2. Navigate to the following folder:

   cd /var/appd/config

   CODE

3. Edit the globals.yaml.gotmpl file with the required configuration.

   vi globals.yaml.gotmpl

   CODE
   1. Update CA certificates for the Ingress controller. The Ingress controller CA certificates are required for the Kubernetes cluster. See Edit the globals.yaml.gotmpl file.
      For Cisco Secure Application, see Configure Domain Name System. 
      
      

      By default, the Ingress controller installs the self-signed certificates.

   2. (Optional) Update the Controller CA certificates.
      The Controller CA certificates are required for the outbound traffic.
   3. (Optional) Disable the self-monitoring for the Controller.
      

      enableClusterAgent: false

      CODE
      See Edit the globals.yaml.gotmpl file.

4. (Optional) Edit the /var/appd/config/secrets.yaml file to update usernames and passwords of theSplunk AppDynamics Services.

   vi secrets.yaml

   CODE
   

   When you install the Splunk AppDynamics service, the secrets.yaml file becomes encrypted.

   See Edit the secrets.yaml.encrypted file.

5. Copy the license files as the license.lic file to the node in the following location.
   

   cd /var/appd/config

   CODE
   
   This license will be automatically used to provision Splunk AppDynamics Services.
   If you do not have the license file at this time, you can apply the license and provision the services later using appdcli.
   
   

   For End User Monitoring, if you are using the Infrastructure-based Licensing model, make sure to specify EUM account and license key in the Administration Console. See Access the Administration Console.

Create a Three-Node Cluster

1. Log in to the primary node console.
2. Verify the boot status of each node of the cluster:
   

   appdctl show boot

   CODE
   

   • Ensure the status of the services in each node appears as Success. Else, restart the virtual machine that failed.
     If it is still failing, you might have to redeploy the virtual machine.
   • Ensure to configure the same time on all the cluster nodes.

3. Run the following command in the primary node and specify the IP address of the peer nodes:

   cd /home/appduser
   appdctl cluster init <Node-2-IP> <Node-3-IP>

   CODE

4. Run the following command to verify the node status:

   appdctl show cluster
   microk8s status

   CODE

   Ensure that the output displays the Running status as true for the nodes that are part of the cluster.

   Sample Output

    NODE           | ROLE  | RUNNING 
   ----------------+-------+---------
    10.0.0.1:19001 | voter | true    
    10.0.0.2:19001 | voter | true    
    10.0.0.3:19001 | voter | true 

   CODE
   

   You must re-login to the terminal if the following error appears:

   Insufficient Permissions to Access Microk8s 

   CODE

Install Services in the Cluster 

1. Log in to the cluster node console.

2. Run the command to install services:

   appdcli start appd [Profile]

   CODE
   appdcli start appd small

   CODE

   
   

   appdcli start appd medium

   CODE

   
   

   
   This command installs the Splunk AppDynamics services. We recommend you to specify the VA profile as same as the profile that you selected to create a virtual machine. See, Virtual Appliance Sizing.

   Sample Output

   NAME               CHART                     VERSION   DURATION
   cert-manager-ext   charts/cert-manager-ext   0.0.1           0s
   ingress-nginx      charts/ingress-nginx      4.8.3           1s
   redis-ext          charts/redis-ext          0.0.1           1s
   ingress            charts/ingress            0.0.1           2s
   cluster            charts/cluster            0.0.1           2s
   reflector          charts/reflector          7.1.216         2s
   monitoring-ext     charts/monitoring-ext     0.0.1           2s
   minio-ext          charts/minio-ext          0.0.1           2s
   eum                charts/eum                0.0.1           2s
   fluent-bit         charts/fluent-bit         0.39.0          2s
   postgres           charts/postgres           0.0.1           2s
   mysql              charts/mysql              0.0.1           3s
   redis              charts/redis              18.1.6          3s
   controller         charts/controller         0.0.1           3s
   events             charts/events             0.0.1           4s
   cluster-agent      charts/cluster-agent      1.16.37         4s
   kafka              charts/kafka              0.0.1           6s
   minio              charts/minio              5.0.14         47s

   CODE
3. Verify the status of the installed pods and service endpoints:
   • Pods: kubectl get pods --all-namespaces

   • Service endpoints: appdcli ping


     +---------------------+---------+
     |  Service Endpoint   | Status  |
     +=====================+=========+
     | Controller          | Success |
     +---------------------+---------+
     | Events              | Success |
     +---------------------+---------+
     | EUM Collector       | Success |
     +---------------------+---------+
     | EUM Aggregator      | Success |
     +---------------------+---------+
     | EUM Screenshot      | Success |
     +---------------------+---------+
     | Synthetic Shepherd  | Success |
     +---------------------+---------+
     | Synthetic Scheduler | Success |
     +---------------------+---------+
     | Synthetic Feeder    | Success |
     +---------------------+---------+
     | AD/RCA Services     | Failed  |
     +---------------------+---------+

     CODE

• When a Virtual Machine restarts, the MySQL service may not automatically restore. To troubleshoot this issue, see Restore the MySQL Service.
• If the EUM pod is failing even after multiple retries, see EUM Health is Failing After Multiple Retries.

Install the Anomaly Detection Services in the Cluster

1. Log in to the cluster node console.

2. Run the command to install services:

   appdcli start aiops small

   CODE

   appdcli start aiops medium

   CODE
3. Verify the status of the installed pods and service endpoints:
   • Pods: kubectl get pods -n cisco-aiops


   • Service endpoints: appdcli ping
     The status of the Anomaly Detection service appears as Success.

See Anomaly Detection.

Sometimes, IOException error occurs when you access Anomaly Detection in the Controller UI. See Troubleshoot Virtual Appliance Issues.

Enable the Cisco Secure Application Service

Complete these steps in the Administration Console to enable the Cisco Secure Application service:

1. Log in to the Administration Console:
   

   https://<controller-hostname>/controller/admin.jsp

   CODE
   

   The default password of the Administration Console is set to welcome.

2. Edit the account to enable Cisco Secure Application and then add the following property:
   

   argento.enabled = true 

   CODE

   This property is added to the tenant that you use to configure agents and log into the UI. For example, if customer1 has your single tenant, then that account needs the argento.enabled = true setting. 

3. Log out of Administration Console, then log into the Controller:
   

   https://<controller-hostname>/controller/

   CODE
4. Create a role with the following permissions:
   • View Cisco Secure Application
   • Configure Cisco Secure Application
5. Assign the role to the administrator user.

Install Cisco Secure Application Services

1. Log in to the cluster node console.
2. Run the following command to install Cisco Secure Application service:
   
   appdcli start secapp small

   CODE

   appdcli start secapp medium

   CODE

   
   

3. Verify the status of the installed pods using the following command:

   kubectl get pods -n cisco-secureapp 

   CODE
4. Log in to the cluster node console.

5. Run the command to install services:

   appdcli start appd [Profile]

   CODE
6. Verify the status of the installed pods and service endpoints:
   

   kubectl get pods -n cisco-secureapp

   CODE

Cisco Secure Application References

Follow the steps to configure Cisco Secure Application:

	Step	Reference
1	For the .NET and Java Agent, you must add node property: enable-secapp-service CODE For the Java Agent, you must be on version >= 24.4.1. For the .NET Agent, you must be on version >= 24.4.0.1.	Install the Java Agent Install the .NET Agent for Linux
2	Extract the SSL certificate for use with the agents. Log in to the cluster node and run the command:  kubectl get secret ingress-cert-secret -n ingress-master -o jsonpath="{.data.tls\.crt}" | base64 --decode > certificate.crt CODE If the cluster is using non default self signed certificate, then copy the existing certificate in the global config location. If defaultCert is false, then run the command:  kubectl get secret custom-ingress-secret -n ingress-master -o jsonpath="{.data.tls\.crt}" | base64 --decode > certificate.crt CODE See globals.yaml.gotmpl file.  Copy this certificate for use with the agent.	Enable SSL for the Java Agent Configuring SSL Variables for .NET Agents
3	Assign roles using the Splunk AppDynamics Administration Console. Assign the Configure Cisco Secure Application account permission to the users who are required to modify configurable fields on the Cisco Secure Application dashboard. Assign View Cisco Secure Application account permissions to users who are required to only monitor the dashboard.	Account Permissions
4	Click on the Security tab in the top navigation bar. Launch the required Splunk AppDynamics Application dashboard using your account, and then click Security on the top pane. This redirects you to the Cisco Secure Application dashboard.	Monitor Application Security Using Cisco Secure Application
5	From the Cisco Secure Application Dashboard navigate to the Applications page, and then set Security Setting as Enabled for the target application. The Security Setting value is set to Inherit by default for all applications that inherit the non-configurable tenant setting of Disabled. To enable security for an application, you must set Security Setting to Enabled.	Monitor Security Status of Applications
6	From the Applicationspage, verify that the application nodes are registered and active. From the Applications page, check the Active Nodes and Registered Nodes fields for the specific application. Ensure that the application nodes are active. If the nodes are not active, then the application security data is not displayed on the dashboard.	Monitor Security Status of Applications
7	From the Libraries page view the risk-sorted libraries of secured applications. The Libraries page displays all the existing libraries of application(s) based on the selected application scope. You can use the risk score to prioritize the remediation task.	Monitor Libraries

For more information, see Getting Started with Cisco Secure Application.

Apply Licenses to Splunk AppDynamics Services

Use appdcli to apply licenses after installingSplunk AppDynamics Services.

1. Log in to the cluster node console.
2. Copy the license files as the license.lic file to the node in the following location.
   

   cd /var/appd/config

   CODE

3. Run the following commands to apply licenses:

   Update the Controller license.

   appdcli license controller license.lic

   CODE

   1. Update the EUM license.

      appdcli license eum license.lic

      CODE
   2. (Optional) If you are using the Infrastructure-based Licensing model, make sure to specify EUM account and license key in the Administration Console. See Access the Administration Console.

For more information, see Virtual Appliance CLI.

Verify the Service Endpoints Paths

The Ingress controller checks the URL of an incoming request and redirects to the respectiveSplunk AppDynamics Service.

Log in to the Controller UI by accessing https://<DNS-Name>or<Cluster-Node-IP>/.

By default, the Controller UI username is set to admin and the password is set to welcome.

Download Splunk AppDynamics Agents

Splunk AppDynamics On-Premises Virtual Appliance supports you to install the AppDynamics agents. Download the agents from Download Portal.

For more information, see:

• Install App Server Agents
• End User Monitoring