You can upgrade a Controller instance using the Enterprise Console. The Enterprise Console simplifies the upgrade process by allowing you to discover and upgrade single Controllers and HA-pairs.
About the Upgrade
- The Enterprise Console supports Controller upgrades (standalone and HA-pair) starting from 4.1 and higher to the latest version. If you are on an earlier version of the Controller, like 3.x or 4.0.x, and would like to upgrade to 4.4.x or the latest version, you would need to use the Controller installer and first upgrade to 4.1. Then use the Enterprise Console to upgrade to 4.4 or the latest version.
- You can upgrade across multiple versions at a time; that is, you do not need to run the upgrade individually for each intermediate version.
- As an optional step, your MySQL version can be upgraded after you upgrade the Controller, through the Enterprise Console GUI or by using the mysql-upgrade job. See Bundled MySQL Database Version for more information.
- If your existing Controller is handling your current load and you are not changing your licensing, you should not need to upgrade your hardware when upgrading the Controller. To determine if you are close to exceeding your Controller's capacity, see Controller performance information in Troubleshoot Controller Issues.
- If you have a license for the older version, the license should work when upgrading the Controller to a new version. However, if you have a temporary license for the old version and now have a new license, the new license will not work on the old Controller. In this case, you should upgrade the Controller to the latest version before applying the license.
- An upgrade results in Controller downtime, but it is not necessary to stop agents during the Controller upgrade.
- If you have configured settings in the domain.xml file, db.cnf or other configuration files manually or by using the Glassfish asadmin utility, verify those changes in the configuration files or Controller Configurations page of the GUI after the upgrade and re-apply any customizations that were not preserved. The Enterprise Console preserves several recommended customizations. After the upgrade, you can find backup copies of common configuration files in the <controller_home>/backup directory.
- Before you start upgrading the Controller, make sure that you are using the correct update order. Remember, if you are upgrading from a pre-4.4 version to 4.4.x or the latest version, you must first manually upgrade any embedded Events Service, then upgrade the Controller.
- Review the latest Release Notes and the release notes for any intermediate versions between the current version of your instance and the version you are targeting to learn about issues and enhancements in those releases.
You may change your Controller profile on the Platform Configurations pages of the Enterprise Console GUI, either before or after you upgrade your Controller. This process is not reversible, and you cannot move from a larger to a smaller profile size.
- Check the Controller's database.log for any errors. You can find the log at
<controller_home>/db/logs/database.log. There should not be any
InnoDB: Errorlines in the log. If any errors are found, please reach out to AppDynamics Support before attempting the upgrade. Upgrading the Controller with a corrupt database may put the Controller in a bad state, with a high recovery time.
- If you changed any Glassfish settings that are not JVM options, note your changes. You may need to configure them after upgrade. The Enterprise Console recognizes and retains many common customizations to the domain.xml, db.cnf, and other configuration files, but is not guaranteed to retain them all. If you have made manual configuration changes to the files, verify the configuration after updating. See Retaining Configuration Changes to learn how to preserve changes.
- The database user password storage mechanism changed in Release 3.8. If upgrading from 3.7.x or earlier, see the Upgrade the Controller page in the 3.8 documentation for considerations related to the database user password.
- Back up the existing Controller following the steps in the next section.
Back up the Existing Controller
The Enterprise Console retains agent data, reports, configuration, and other types of data through an upgrade, including a copy of commonly modified configuration files under <controller_home>/backup. Nevertheless, to mitigate the risk of data loss in the event of an unexpected failure, be sure to back up the existing installation directory before applying an upgrade to the Controller.
The Enterprise Console does not back up MySQL data. You need to back up the data before upgrading standalone installations.
If the upgrade does not finish successfully for any reason, see Troubleshooting the Upgrade for more information.
To back up the Controller instance
- Stop the Controller application server and database.
For Linux, run:
For Windows, run:
- Back up the Controller home by copying the entire Controller home directory to a backup location. Note the following points:
- If the data home for the Controller is not under the Controller directory, be sure to back up the database directory as well.
- If it's not possible to back up the entire data set, you can selectively back up the most important tables. Use the Metadata Backup SQL script described and attached to the Controller Data Backup and Restore page.
- Restart your Controller after completing the backup and before upgrading.
Troubleshooting the Upgrade
If the upgrade does not succeed for any reason, the Enterprise Console does not roll back changes on disk. This give you an opportunity to diagnose and troubleshoot the issue before reattempting the installation or upgrade.
To troubleshoot the issue, check the installation log at platform-admin/logs/platform-admin-server.log. The Controller server.log file may contain additional information.
The Enterprise Console provides a feature to resume the upgrade from the last point of failure (checkpoint). The application creates a checkpoint at several stages during installation. If the installation fails, resuming from checkpoint would skip all the prior successfully completed stages and restart the installation from the beginning of the specific stage where the last point of failure occurred.
If for some reason, the upgrade from the last checkpoint is not successful, you may retry the upgrade from beginning. Simply click Retry instead of Resume from the checkpoint. However, please note that retrying from the beginning after a failed upgrade may overwrite your customizations to db.cnf and domain.xml.
A common upgrade issue involves the upgrade process timing out. The Enterprise Console attempts to restart the Controller and database after applying an update or installation. For large databases and depending on the system resources, this can take a considerable amount of time. If the Enterprise Console cannot finish starting up the Controller within a set time-out period (30 minutes by default), the operation will fail.
You can increase the default time out period for system startup. The timeouts are defined in platform-admin/archives/controller/<version>/playbooks/controller.groovy. You can update the controllerStartRetryTimeout = 45 * 60 value, which is in minutes, then retry the upgrade from the checkpoint.