Related pages:

You can upgrade a Controller instance using the Enterprise Console. The Enterprise Console simplifies the upgrade process by allowing you to discover and upgrade single Controllers and HA-pairs.

About the Upgrade

  • The Enterprise Console supports Controller upgrades (standalone and HA-pair) starting from 4.4.x and higher, to the latest version. Use the following table to determine your course of action based on your circumstances:

    If your current Controller version is...Controller version you want to upgrade to...Actions to take...
    < 4.4.x (such as: 4.1.x, 4.2.x, 4.3.x)Controller version 4.5.x or the latest version
    1. Open a support help desk ticket.
    2. Support will provide you a 4.3.8 installer for you to use in the upgrade process.
    3. Use the Controller installer and upgrade the Controller to version 4.3.8.
    4. Then, use Enterprise Console version 4.4.3 to upgrade the Controller from version 4.3.8 to version 4.4.3.
    5. Finally, use Enterprise Console version 4.5.x to upgrade the Controller from version 4.4.3 to version 4.5.x.
    >= 4.4.xController version 4.5.17 or prior 4.5.x versions 
    1. Access the download.appdynamics.com site to download the Controller version.
    2. Use the Enterprise Console to upgrade from your existing Controller version to the Controller version you want.
  • Discovering and upgrading a Controller from 4.1 and higher to the latest version will not upgrade the Events Service automatically. This must be done by a separate upgrade job.
  • You can choose which version you would like to upgrade the Controller to as long as the Enterprise Console is aware of that version. This means that you can upgrade the Controller to any intermediate version or to the latest version as long as the Enterprise Console installer has been run for those versions. However, you cannot upgrade the Controller to an older version.
  • If you have a license for the older version, the license should work when upgrading the Controller to a new version. However, if you have a temporary license for the old version and now have a new license, the new license will not work on the old Controller. In this case, you should upgrade the Controller to the latest version before applying the license. 
  • An upgrade results in Controller downtime, but it is not necessary to stop agents during the Controller upgrade.

  • Starting in 4.5, during a Controller discover and upgrade scenario, the Enterprise Console expects a .passwordfile file to be present in the Controller home directory. The Enterprise Console reads this password and validates it against the Controller. Once the upgrade is complete, the Enterprise Console removes the file, and stores the password in its encrypted database.

    If you change the Glassfish Admin Password manually, you also need to update it in the Enterprise Console Controller Settings.

Before Upgrading

  • Before you start upgrading the Controller, make sure that you are using the correct update order. Remember, if you are upgrading from a pre-4.1 version to 4.4.x or the latest version, you must first manually upgrade any embedded Events Service, then upgrade the Controller.
  • Review the latest Release Notes and the release notes for any intermediate versions between the current version of your instance and the version you are targeting to learn about issues and enhancements in those releases. 

  • Check the most recent Controller System Requirements and Troubleshoot Controller Issues to review your Controller's current workload and determine whether you need to change your performance profile and increase your hardware resources, if necessary.

    You may change your Controller profile on the Platform Configurations pages of the Enterprise Console GUI, either before or after you upgrade your Controller. This process is not reversible, and you cannot move from a larger to a smaller profile size. 

  • Check the Controller's database.log for any errors. You can find the log at <controller_home>/db/logs/database.log. There should not be any InnoDB: Error lines in the log. If any errors are found, please reach out to AppDynamics Support before attempting the upgrade. Upgrading the Controller with a corrupt database may put the Controller in a bad state, with high recovery time.
  • If you changed any Glassfish settings that are not JVM options, note your changes. You may need to configure them after an upgrade. The Enterprise Console recognizes and retains many common customizations to the domain.xml, db.cnf, and other configuration files, but is not guaranteed to retain them all. If you have made manual configuration changes to the files, verify the configuration after updating. See Retaining Configuration Changes to learn how to preserve changes.
  • If you uninstall the Enterprise Console that was previously managing the Controller and use a new Enterprise Console instance to discover and upgrade the Controller, you need to first manually create the .passwordfile file in order for the Enterprise Console to continue with the discover and upgrade process. You can create the file in the Controller home directory, and add the AS_ADMIN_PASSWORD=<controllerRootUserPassword> value in it.
  • When performing an upgrade while enabling HTTPS, check that the hostname does not start with a digit. The upgrade will fail if the Enterprise Console hostname starts with a digit due to a JDK limitation with the DNS name in CN/SAN.
  • The database user password storage mechanism changed in Release 3.8. If upgrading from 3.7.x or earlier, see the Upgrade the Controller page in the 3.8 documentation for considerations related to the database user password.
  • Back up the existing Controller following the steps in the next section.

Back up the Existing Controller

The Enterprise Console retains agent data, reports, configuration, and other types of data through an upgrade, including a copy of commonly modified configuration files under <controller_home>/backup. Nevertheless, to mitigate the risk of data loss in the event of an unexpected failure, be sure to back up the existing installation directory before applying an upgrade to the Controller.

The Enterprise Console does not back up MySQL data. You need to back up the data before upgrading standalone installations.

If the upgrade does not finish successfully for any reason, see Troubleshooting the Upgrade for more information. 

To back up the Controller instance
  1. Stop the Controller application server and database.

    • For Linux, run: 

      platform-admin.sh stop-controller-appserver

    • For Windows, run:

      platform-admin.exe cli stop-controller-appserver --with-db
  2. Back up the Controller home by copying the entire Controller home directory to a backup location. Note the following points: 
    • If the data home for the Controller is not under the Controller directory, be sure to back up the database directory as well. 
    • If it's not possible to back up the entire data set, you can selectively back up the most important tables. Use the Metadata Backup SQL script described and attached to the Controller Data Backup and Restore page. 
  3. Restart your Controller after completing the backup and before upgrading.

After Upgrading

  • If you have configured settings in the domain.xml file, db.cnf or other configuration files manually or by using the Glassfish asadmin utility, verify those changes in the configuration files or Controller Configurations page of the GUI after the upgrade and re-apply any customizations that were not preserved. The Enterprise Console preserves several recommended customizations. After the upgrade, you can find backup copies of common configuration files in the <controller_home>/backup directory.
  • As an optional step, your MySQL version can be upgraded after you upgrade the Controller, through the Enterprise Console GUI or by using the mysql-upgrade job. See Bundled MySQL Database Version for more information.

Troubleshooting the Upgrade

If the upgrade does not succeed for any reason, the Enterprise Console does not roll back changes on disk. This gives you an opportunity to diagnose and troubleshoot the issue before reattempting the installation or upgrade. 

To troubleshoot the issue, check the installation log at platform-admin/logs/platform-admin-server.log. The Controller server.log file may contain additional information. 

Resuming from Checkpoint

The Enterprise Console provides a feature to resume the upgrade from the last point of failure (checkpoint). The application creates a checkpoint at several stages during installation. If the installation fails, resuming from checkpoint would skip all the prior successfully completed stages and restart the installation from the beginning of the specific stage where the last point of failure occurred.

You can also resume a failed Controller job from the CLI by passing the flag useCheckpoint=true as an argument after --args in your command.

If for some reason, the upgrade from the last checkpoint is not successful, you may retry the upgrade from the beginning. Simply click Retry instead of Resume from the checkpoint. However, please note that retrying from the beginning after a failed upgrade may overwrite your customizations to db.cnf and domain.xml.

Timing Out

A common upgrade issue involves the upgrade process timing out. The Enterprise Console attempts to restart the Controller and database after applying an update or installation. For large databases and depending on the system resources, this can take a considerable amount of time. If the Enterprise Console cannot finish starting up the Controller within a set time-out period (30 minutes by default), the operation will fail.

You can increase the default time out period for system startup. The timeouts are defined in platform-admin/archives/controller/<version>/playbooks/controller.groovy. You can update the controllerStartRetryTimeout = 10 * 60 seconds = 10 minutes, and then retry the upgrade from the checkpoint.

When the Controller upgrade is complete, audit reports may not work immediately. The audit database table is getting migrated only after the upgrade process and the migration takes at least an hour to complete. If audit reports are run before completing the migration process, audit table migration messages are logged in the server.log file. No actions are required, try running the audit reports again after an hour.