AWS Services

The below table lists mandatory and optional managed AWS services that are used by Usage Engine

Assets

Stateful Data Assets - To Include in Backups

When running in AWS, Usage Egnine stores data persistently in RDS, EFS and optionally MemoryDB. All these services support encryption at rest. The default Terraform templates provided as part of the product are configured to use encryption at rest. MemoryDB is configured by the customer. It is recommended to configure it with encryption at rest if sensitive information will be stored in it.

Sensitive Data

Data can be separated in two categories: system configuration data and solution runtime data.

System Configuration Data

System data is stored in the system database in RDS. It includes things like users and roles with login credentials. Login credentials are stored in a secure way and cannot be reverse engineered. Other information in this category controls how the system is configured. The data is not sensitive from a personal integrity perspective but is from a general IT security perspective. It is thus recommended to have encryption at rest enabled for the system database.

Solution Runtime Data

As workflows are executed in the system and process payload they persist intermittent data. This data is stored in various locations:

• Transaction data - in system database in RDS
• Aggregation session - on EFS or in MemoryDB
• Duplicate detection caches - on EFS
• Inter workflow persisted transmission buffers - on EFS
• Error correction data - in system database in RDS (Data Veracity subsystem) or on EFS (older ECS subsystem)

Depending on the characteristics on the solution, this information can contain sensitive information about subscribers etc. Thus it is recommended to use encryption at rest for EFS and MemoryDB as soon as there is some data with personal integrity being processed in the workflows.

Data Location

EFS Disk Content

Solutions that use disk persisted data is storing this on a mounted EFS disk resource. Using the provided Terraform template, the EFS resource is given the name "[EKS-CLUSTER-ANME]-efs-disk". It is mounted into the pods at the path "/opt/mz/persistent" which is the path where executing workflows can access the storage. 

The following features use or can use disk based storage to persist data:

• Aggregation Aggregation (3.0)

• Duplicate UDR Detection Duplicate UDR (3.0)

• Inter Workflow Inter Workflow (3.0)

RDS Managed database

The provided Terraform installation templates sets up a PostgreSQL database in RDS to be used as the system database. The RDS resource gets the name "[EKS-CLUSTER-ANME]-db".

MemoryDB Redis database 

Some features can use a Redis compatible database to store state. In AWS the MemoryDB for Redis service can be used for this. Access to this is configured in:

• Redis Profile Redis (3.0)

and can be consumed by:

• Aggregation Profile Aggregation (3.0)
• Distributed Storage Profile Distributed Storage (3.0)

Stateless Assets - To Manage as Infrastructure as Code

The AWS infrastructure should be managed as IaC using Terraform, CloudFormation or similar tool.  Usage Engine comes with templates written in Terraform that will setup a basic environment. The default configuration of these templates setup the following resources, which are sufficient to run a standard Usage Engine system with solutions that are not particularly demanding on resources. 

• EFS Mount Target
• EFS Security Group
• KMS Key
• KMS Alias
• RDS DB Instance (Postgres)
• RDS DB Subnet Group
• RDS DB Access Security Group
• SSM RDS Parameters
• DB User
• DB Password
• Route53 DNS Zone
• Route53 DNS NS Record
• ACM Certificate
• Route53 Cert Validation Record
• ACM Certificate Validation
• EKS Cluster
• VPC Cluster Endpoint
• Node Group
  • Minimum three nodes m5.Large
• CloudWatch cluster logging
• MZ Platform Application Helm chart
• MZ ECD Solution Helm charts