Back Up Using the S3 Compatible Storage of MinIO

You can back up the Virtual Appliance data to MinIO S3-compatible storage. Follow these steps to back up the Virtual Appliance data:

Set Up MinIO S3 Compatible Storage

Before you back up the Virtual Appliance data using MinIO, follow these steps to set up the MinIO server:

Ensure that the 9000 port is available for the MinIO server.

1. Download the MinIO server for the Linux OS.
2. Grant the execute permission to the MinIO file.
   

   chmod +x minio

   CODE
3. Create a directory to store CA certificates.
   

   mkdir -p ~/.minio/certs/CAs
   cd ~/.minio/certs

   CODE
4. Generate Certificate Signing Request and private key for the MinIO server:
   

   openssl genpkey -algorithm RSA -out private.key
   openssl req -new -key private.key -out cert.csr
   openssl x509 -req -days 365 -in cert.csr -signkey private.key -out public.crt
   ./minio server --address :9000 /mnt/data

   CODE

Configure MinIO Certificates on the Virtual Appliance

You require the CA certificates of MinIO on the Virtual Appliance to establish an SSL connection between the MinIO server and store snapshots.

Copy public.crt of MinIO to the Virtual Appliance cluster.

/var/appd/config/postgres-s3-ca.crt

/var/appd/config/mysql-s3-ca.crt

Elasticsearch requires the JKS format so convert the public.crt to JKS file.

1. Run the following command to generate the JKS file from the public.crt:
   

   keytool -importcert -keystore es-s3-ca.jks -storepass changeit -file public.crt -alias minio

   CODE
2. Copy the es-s3-ca.jks file to the following location:
   

   /var/appd/config

   CODE
3. Configure the truststore password in the secrets.yaml. For example, changeit.
   See Edit the secrets.yaml.encrypted file.
   
   

   • Access the MinIO Console to generate access and secret keys.
   • Specify the storepassword of es-s3-ca.jks as the trustStorePassword in secrets.
   • To sync the changes in secret.yaml, run the appdcli sync command:
   
   

   # S3 secrets
   ##
   backup:
       elasticsearch:
           s3:
               access_key: WTFaWld3c1NHWWNOSzBWeDlXaHI=
               secret_key: TWtJY0NweVdIcEhJV0pxdzdIWkRnVld3dzAxODRqbjhnS1pnZHozag==
               trustStorePassword: changeit

   CODE

Configure the MinIO Endpoint in the globals.yaml.gotmpl File

1. Create a repository to back up the data in the S3 compatible storage.
   
   

   You can create repository only for the Elasticsearch backup. By default, the globals.yaml.gotmpl file uses the repo2 repository for MySQL and PostgreSQL. 

   
   

   appdcli run es_backup create-s3-repo virtual-appliance-repo-minio

   CODE
2. Enable the s3 section for the datastores to back up using the MinIO server. See globals.yaml.gotmpl file.
   
   # Backup config for datastores
   backup:
     elasticsearch:
       s3:
         enabled: true
         endpoint: https://s3-endpoint
         protocol: https
         bucket: es-bucket
         region: us-east-1     
         {{ if isFile "/var/appd/config/es-s3-ca.jks" }}
         certFile: {{ readFile "/var/appd/config/es-s3-ca.jks" | b64enc | quote }}
         {{ end }}

   CODE

   
   

   # Backup config for datastores
   backup:
     postgresql:
       s3:
         enabled: true
         repoName: repo2
         endpoint: https://10.0.203.142:9000
         bucket: postgresql-bucket
         region: us-east-1
         {{ if isFile "/var/appd/config/postgres-s3-ca.crt" }}
         certFile: {{ readFile "/var/appd/config/postgres-s3-ca.crt" | b64enc | quote }}
         {{ end }}

   CODE

   
   

   
   

   

   Do not specify the HTTPS endpoint because MySQL currently supports only HTTP endpoint. 

   
   

   # Backup config for datastores
   backup:  
     mysql:
       s3:
         enabled: true
         repoName: repo2 
         endpoint: http://10.0.203.142:9000
         bucket: mysql-bucket
         region: us-east-1
         {{ if isFile "/var/appd/config/mysql-s3-ca.crt" }}
         certFile: {{ readFile "/var/appd/config/mysql-s3-ca.crt" | b64enc | quote }}
         {{ end }}

   CODE

   
   

   
   

Back Up the Virtual Appliance Data in S3 Compatible Storage

Snapshots capture the state of the datastore and can be used to restore data. You can store multiple snapshots in a single repository. The following commands stores the snapshots in the virtual-appliance-repo-minio repository.

Create a snapshot for each datastore:

appdcli run es_backup create-snapshot <repo-name> <snapshot-name>

appdcli run es_backup create-snapshot virtual-appliance-repo-minio es-snapshot

appdcli run pg_backup create-snapshot <repo-name> <snapshot-name>

appdcli run pg_backup create-snapshot repo2 pg-snapshot

appdcli run mysql_backup create-snapshot <repo-name> <snapshot-name>

appdcli run mysql_backup create-snapshot repo2 mysql-snapshot

Restore Snapshots from the MinIO server

Use the snapshots that you have stored in MinIO server to restore the datastores.

You may face issues while restoring a large backups. In such cases, bring the Controller down.

helm delete controller -n cisco-controller 

CODE

1. List the snapshots in the repository:
   
   appdcli run es_backup list-snapshot <repo-name>

   CODE

   Example

   appdcli run es_backup list-snapshot virtual-appliance-repo-mino 

   CODE

   appdcli run pg_backup list-snapshot <repo-name>

   CODE

   Example

   appdcli run pg_backup list-snapshot repo2

   CODE

   
   

   appdcli run mysql_backup list-snapshot <repo-name>

   CODE

   Example

   appdcli run mysql_backup list-snapshot repo2

   CODE

   
   

2. Restore the datastore by one of the snapshots:
   
   appdcli run es_backup restore-snapshot <repo-name> <snapshot-name>

   CODE

   Example

   appdcli run es_backup restore-snapshot virtual-appliance-repo-mino es_snapshot

   CODE

   appdcli run pg_backup restore-snapshot <repo-name>

   CODE

   Example

   appdcli run pg_backup restore-snapshot repo2 pg_snapshot

   CODE

   
   

   appdcli run mysql_backup restore-snapshot <repo-name>

   CODE

   Example

   appdcli run mysql_backup restore-snapshot repo2 mysql_snapshot

   CODE

   
   

After the restore is successful, if the Controller is down, run the following command:

appdcli sync appd <profile>

Delete the Snapshots from the MinIO S3 Compatible Storage

If you no longer require the snapshot, you can delete the snapshot from the MinIO S3 Compatible Storage.

appdcli run es_backup delete-snapshot <repo-name> <snapshot-name>

appdcli run es_backup delete-snapshot virtual-appliance-repo-mino es_snapshot

appdcli run pg_backup delete-snapshot <repo-name>

appdcli run pg_backup delete-snapshot repo2 pg_snapshot

appdcli run mysql_backup delete-snapshot <repo-name>

appdcli run mysql_backup delete-snapshot repo2 mysql_snapshot