This section describes the standard upgrade procedure for upgrading Usage Engine on Oracle Cloud Infrastructure. This procedure should be used to ensure that configurations, persisted data (in the file system or database), and system properties are properly migrated. Workflows will be interrupted shortly at the restart of the ECDs. These instructions will also describe how the timing of the restart of ECDs running non-scalable real-time workflows can be controlled by enabling manual upgrade.

Preparing ECDs

Backup the database by following these steps:

1. List the databases and locate the one used for Usage Engine:

oci psql db-system-collection list-db-systems --compartment-id <compartment_OCID> --all

2. Perform a backup of the database by running the following command:

oci psql backup create --compartment-id <compartment_OCID> --db-system-id <db_system_OCID> --display-name <backup_display_name>

3. Check if the backup was created successfully by running this command:

oci psql backup-collection list-backups --compartment-id <compartment_OCID> --display-name <backup_display_name> --all

4. Get details about a backup by running this command:

oci psql backup get --backup-id <backup_OCID>

The next step is to do a backup of the file system used.

oci fs snapshot create --file-system-id <file_system_OCID> --name "<snapshot_name>" --expiration-time <2023-09-15T20:30Z>

After creation, snapshots are accessible under the root directory of the file system at .snapshot/<snapshot_name>.

Restore Database Backup

To restore the backup, run these commands:

oci psql db-system restore --db-system-id <db_system_OCID> --backup-id <backup_OCID>

Restore File System Snapshot

To restore a file system snapshot you need to access the snapshots via a Kubernetes Pod.

You must know the mount target IP address prior to mounting the file system onto the Pod. Before running the Pod, retrieve the mount target IP address from PV output:

 kubectl get pv <pv-name> -o jsonpath='{.spec.csi.volumeHandle}'|awk -F: '{print $2}'

Ensure to save the IP address as it will be used as input to mount the file system onto the Pod later.

To run a Kubernetes Pod, run this command:

kubectl run nfscli --rm --tty -i --restart='Never' --namespace uepe --image oraclelinux --privileged=true --command -- bash

You will see the command prompt if the Pod created and running. The command prompt indicates that you have logged in to the Pod successfully.

user1@user1-MacBook-Pro ~ % kubectl run nfscli --rm --tty -i --restart='Never' --namespace uepe --image oraclelinux --privileged=true --command -- bash
If you don't see a command prompt, try pressing enter.
[root@nfscli /]# 
[root@nfscli /]# 

On the pod, download and install nfs-utils. These are libraries and executables needed to access the File System Storage.

[root@nfscli /]# yum -y install nfs-utils 

On the pod, create the mount directory. This is the directory where the mounted file system is located.

[root@nfscli /]# mkdir -p /mnt/uepe

On the pod, mount the file system with the mount target IP address that you retrieved and saved prior to running the pod.

[root@nfscli /]# mount -o nolock <IP address of the mount target>:/uepe /mnt/uepe

On the pod, list the mount directory.

[root@nfscli /]# ls -al /mnt/uepe/
total 3
drwxrwsr-x. 9 root 6000  7 Aug 14 06:51 .
drwxr-xr-x. 1 root root 18 Aug 15 02:34 ..
drwxrwsr-x. 5 root 6000  3 Aug 15 02:34 .snapshot
drwxrwsr-x. 2 6000 6000  0 Aug 14 03:43 3pp
drwxrwsr-x. 3 6000 6000  1 Aug 15 02:00 backup
drwxrwsr-x. 2 6000 6000  0 Aug 14 03:43 jni
drwxrwsr-x. 2 root 6000  0 Aug 14 03:43 keys
drwxrwsr-x. 5 6000 6000  3 Aug 14 06:57 log
drwxrwsr-x. 3 6000 6000  1 Aug 14 05:12 pico-cache
drwxrwsr-x. 2 6000 6000  0 Aug 14 03:43 storage

On the pod, list the available snapshots. These directories listed under .snapshot/ are the <snapshot_name> provided during backup of the file system in previous section.

[root@nfscli /]# ls -al /mnt/uepe/.snapshot/
total 3
drwxrwsr-x. 5 root 6000 3 Aug 15 02:35 .
drwxrwsr-x. 9 root 6000 7 Aug 14 06:51 ..
drwxr-xr-x. 2 root 6000 1 Aug  8 07:26 snapshot-aug-1
drwxr-xr-x. 2 root 6000 2 Aug  8 10:01 snapshot-aug-2
drwxrwsr-x. 9 root 6000 7 Aug 14 06:51 snapshot-aug-3

To restore these snapshots within the Pod, copy these snapshots to the destination directory.

[root@nfscli /]# cp -rp /mnt/uepe/.snapshot/<snapshot_name>/* <destination_directory_name>

For example, to restore “snapshot-aug-3“ snapshot to platform volume mount directory, follow these steps:

1. Clean up the existing platform volume mount directory.

[root@nfscli /]# rm -rf /mnt/uepe/*

2. Copy data from “snapshot-aug-3“ to platform volume mount directory.

[root@nfscli /]# cp -rp /mnt/uepe/.snapshot/snapshot-aug-3/* /mnt/uepe/

3. Unmount the file system if restore completed successfully.

   Code Block
   [root@nfscli /]# umount /mnt/uepe/

4. Exit the Pod

[root@nfscli /]# exit