AWS Services
The below table lists mandatory and optional managed AWS services that are used by
Assets
Stateful Data Assets - To Include in Backups
When running in AWS, stores data persistently in RDS, EFS and optionally MemoryDB. All these services support encryption at rest. The default Terraform templates provided as part of the product are configured to use encryption at rest. MemoryDB is configured by the customer. It is recommended to configure it with encryption at rest if sensitive information will be stored in it.
Sensitive Data
Data can be separated in two categories: system configuration data and solution runtime data.
System Configuration Data
System data is stored in the system database in RDS. It includes things like users and roles with login credentials. Login credentials are stored in a secure way and cannot be reverse engineered. Other information in this category controls how the system is configured. The data is not sensitive from a personal integrity perspective but is from a general IT security perspective. It is thus recommended to have encryption at rest enabled for the system database.
Solution Runtime Data
As workflows are executed in the system and process payload they persist intermittent data. This data is stored in various locations:
- Transaction data - in system database in RDS
- Aggregation session - on EFS or in MemoryDB
- Duplicate detection caches - on EFS
- Inter workflow persisted transmission buffers - on EFS
- Error correction data - in system database in RDS (Data Veracity subsystem) or on EFS (older ECS subsystem)
Depending on the characteristics on the solution, this information can contain sensitive information about subscribers etc. Thus it is recommended to use encryption at rest for EFS and MemoryDB as soon as there is some data with personal integrity being processed in the workflows.
Data Location
EFS Disk Content
Solutions that use disk persisted data is storing this on a mounted EFS disk resource. Using the provided Terraform template, the EFS resource is given the name "[EKS-CLUSTER-ANME]-efs-disk". It is mounted into the pods at the path "/opt/mz/persistent" which is the path where executing workflows can access the storage.
The following features use or can use disk based storage to persist data:
Aggregation Aggregation (3.0)
Duplicate UDR Detection Duplicate UDR (3.0)
Inter Workflow Inter Workflow (3.0)
RDS Managed database
The provided Terraform installation templates sets up a PostgreSQL database in RDS to be used as the system database. The RDS resource gets the name "[EKS-CLUSTER-ANME]-db".
MemoryDB Redis database
Some features in can use a Redis compatible database to store state. In AWS the MemoryDB for Redis service can be used for this. Access to this is configured in:
- Redis Profile Redis (3.0)
and can be consumed by:
- Aggregation Profile Aggregation (3.0)
- Distributed Storage Profile Distributed Storage (3.0)
Stateless Assets - To Manage as Infrastructure as Code
The AWS infrastructure should be managed as IaC using Terraform, CloudFormation or similar tool. comes with templates written in Terraform that will setup a basic environment. The default configuration of these templates setup the following resources, which are sufficient to run standard
system with solutions that are not particularly demanding on resources.
- EFS Mount Target
- EFS Security Group
- KMS Key
- KMS Alias
- RDS DB Instance (Postgres)
- RDS DB Subnet Group
- RDS DB Access Security Group
- SSM RDS Parameters
- DB User
- DB Password
- Route53 DNS Zone
- Route53 DNS NS Record
- ACM Certificate
- Route53 Cert Validation Record
- ACM Certificate Validation
- EKS Cluster
- VPC Cluster Endpoint
- Node Group
- Minimum three nodes m5.Large
- CloudWatch cluster logging
- MZ Platform Application Helm chart
- MZ ECD Solution Helm charts
Add Comment