Verify the Migration Status

If any of the commands fail to copy the data to the target, run those commands again.

Check the status of the index in the following files in the same directory where the jar is located:

migration_output*success.csv
migration_output*failure.csv

Example filenames:

metadata_migration_output__ALL_SUCCESS_1691428484168.csv
migration_output_Final__ALL_SUCCESS_1691021040147.csv
metadata_migration_output_HAS_FAILURE_1691428484168.csv
migration_output__HAS_FAILURE_1691428484168.csv

After the migration is successful, enable job compaction for the INDEX_COMPACTION_PARENT_JOB index:

curl -H 'Content-type: application/json' -XPOST 'http://<ELB_hostname_of_Events_Service_24.x>:<ElasticSearch_Port>/job_framework_job_details_v1/_update_by_query' -d '{
 "query": {
  "term": {
   "name": "INDEX_COMPACTION_PARENT_JOB"
  }
 },
 "script": {
  "inline": "ctx._source.jobDataMap.enabled = true",
  "lang": "painless"
 }
}'

CODE

Reset Migration

To undo any changes during migration and start afresh, perform the following steps:

Delete dataMigration.db or rename/move existing db file.
Delete metadataIndices_history.json or rename/move metadataIndices_history.json.
Repeat the migration steps.

To redo the migration for any particular index, perform the following steps for the respective index:

Connect to SQLite from the command line:
```
sqlite3 <Your_Database_name>
```
CODE

Run following command to delete status of index

DELETE FROM tasks WHERE candidate_index = <index_name> and task_status = "SUCCESS";

SQL

Run the data migration again for that index. See Migrate Events Service Data.

Useful commands

To get information regarding the migration utility, run the following command:

java -jar analytics-on-prem-es2-es8-migration-LATESTVERSION-exec.jar --help

CODE

SQL Commands

You can optionally determine the status of the migration using the dataMigration.db instead of the logs. The migration database (dataMigration.db) stores the migration details for customer indices in the following tables:

WORKFLOW — Contains each iteration of the data migration job. Every iteration has a unique workflow ID.
CANDIDATES — Contains the list of indices that the utility has detected for migration.
TASKS — Tracks each task. It contains common and index specific tasks, the common task has the candidate_index value = "ALL".
REINDEX_TASK_STATUS — Contains the re-indexed state of each candidate's index.

Note that the dataMigration.db does not store any status for metadata indices. To know the details of metadata indices migration, see metadataIndices_history.json.

To monitor the migration status using SQLite, run the following command:

sqlite3 dataMigration.db (dataMigration.db will be generated in the same folder where the jar file resides, hence you must run this either on the same folder or provide absolute path).
select * from tasks;

CODE

To obtain the indices that have not registered any failures during migration:
- Total number of documents to be re-indexed or successfully re-indexed.
- Created documents in the Events Service 24.x.
- Updated documents in the Events Service 24.x.
```
SELECT candidate_index, total, created, updated  from REINDEX_TASK_STATUS where completed == 1 and is_failed = 0 
```
  SQL
To obtain the indices that have registered any failures during migration::
- Total number of documents to be re-indexed or successfully re-indexed.
- Created documents in the Events Service 24.x.
- Updated documents in the Events Service 24.x.
```
SELECT candidate_index, total, created, updated  from REINDEX_TASK_STATUS where completed == 1 and is_failed = 1 
```
  SQL

To view the status of the documents where re-indexing is in progress:

SELECT candidate_index, total, created, updated  from REINDEX_TASK_STATUS where completed == 0

SQL

You may come across some errors during migration. See Troubleshoot Migration Issues and resolve them.