Bootstrapping System Certificates and Secrets - AWS(3.3)

Prior to installation, the system must be bootstrapped with certain system credentials and certificates. There are a few alternative ways to perform this task.

System Credentials

The system credentials can be provided to the system in the following ways:

Manually creating a Kubernetes Secret in advance.
Providing the information as Helm values.
Letting the system randomly generate the information.

These options are explained in the following sections.

Necessary system credentials for Secret or Helm, respectively, are listed in the table. These credentials for database and system level API need to be provided before installation can complete:

Secret Parameter	Helm Parameter	Description
jdbcUser	platform.db.jdbcUser	User to authenticate to system database.
jdbcPassword	platform.db.jdbcPassword	Password to authenticate to system database.
mzownerPassword	oracle.mzownerPassword postgres.mzownerPassword	Schema owner password to authenticate to system database.
postgresqlPassword	postgres.adminPassword	Postgres database admin password.
oraclePassword	oracle.adminPassword	Oracle database admin password.
tlsKeystorePassword	platform.tls.key.storepassword	System keystore password. Also used in cert-manager requests.
tlsKeyPassword	platform.tls.key.password	System key password. Also used in cert-manager requests.

Kubernetes Secret

Credentials can be written into a Secret object named env-secrets prior to installation.

Example - Secret object

$ kubectl create secret generic env-secrets -n <namespace> \ 
--from-literal=jdbcUser=jdbcuser \
--from-literal=jdbcPassword=jdbcpassword \
--from-literal=mzownerPassword=mzownerpassword \
--from-literal=postgresqlPassword=postgrespassword \
--from-literal=tlsKeystorePassword=keystorepassword \
--from-literal=tlsKeyPassword=keypassword

Helm Values

Credentials can also be provided through values to Helm, by providing them in values.yaml or by passing them on the command line.

Example - Helm credentials

$ helm install <release_name> ./usage-engine-private-edition --wait --timeout=5m --namespace <namespace> \
--set platform.db.jdbcUser=<user> \
--set platform.db.jdbcPassword="<password>" \
--set postgres.mzownerPassword="<password>" \
--set postgres.adminPassword="<password>" \
--set platform.tls.key.storepassword="<password>" \
--set platform.tls.key.password="<password>"

Note!

Be aware that storing secret credentials in a values.yaml file in version control is not secure! If you still need to do this you should consider using tools like https://github.com/mozilla/sops.

Random

If neither Secret nor Helm values are used to provide credentials, the required values are automatically populated with random data. For users with access rights, the randomized credentials can be retrieved by doing:

$ kubectl get secret/env-secrets -n <namespace> -o yaml

The values are stored base64 encoded and need to be decoded to be viewed.

Example - Decoding

$ kubectl get secrets/env-secrets -n <namespace> --template={{.data.jdbcPassword}} | base64 -d

Certificates

Usage Engine requires a certificate to provide encryption on the public (external) interfaces. This certificate can be provided through a few different methods; cert-manager, Secret, and Disk Based Keystore.

The method used is controlled through the following parameters in values.yaml:

Example - Configuration of certificate method in values.yaml

platform:
	tls:
		enabled: true
    	cert:
      		# Public certificate can be provided through one of methods: 'secret', 'certManager' or 'key' (not recommended)
      		public: certManager

Kubernetes cert-manager

The most automated and secure way to provide certificates to the system is to use the Kubernetes tool cert-manager.

Follow the instruction in the link below to install cert-manager:

https://cert-manager.io/docs/installation/

Cert-manager must be backed by a certificate authority (CA) to sign the certificates. Once configured with a CA, cert-manager will automatically sign and renew certificates for the system as needed. Configuring cert-manager with a CA is done be defining an Issuer or ClusterIssuer resource and referring that one from values.yaml.

Example - Configuration in values.yaml to enable cert-manager based keystore

platform:
  tls:
    cert:
      public: certManager
    certManager:
      public: 
        issuer:
          name: letsencrypt-prod
          kind: ClusterIssuer
          domain: uepe.example.com

For a list of supported CA Issuer or ClusterIssuer types, see https://cert-manager.io/docs/configuration/

The following example configures a ClusterIssuer CA towards the free public CA provided Letsencrypt in an AWS deployment.

Example - ClusterIssuer CA configuration

apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    # The ACME server URL
    server: https://acme-v02.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: admin@example.com
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    # example: cross-account zone management for example.com
    # this solver uses ambient credentials (i.e. inferred from the environment or EC2 Metadata Service)
    # to assume a role in a different account
    - selector:
        dnsZones:
          - "uepe.example.com"
      dns01:
        route53:
          hostedZoneID: XXXXXXXXXXXX
          region: eu-west-1

Hint!

For troubleshooting failed certificate provisioning with cert-manager, there is an excellent troubleshooting guide available:

https://cert-manager.io/docs/faq/troubleshooting/

Secret

If you do not want to automate the certificate provisioning with cert-manager, you can instead manually install a public certificate in a Secret and refer the installation to that Secret.

Example - Configuration in values.yaml to enable Secret based keystore

platform:
	tls:
		cert:
      		public: secret
		secret:
	    	public:
        		name: mz-cert
        		keystore:
          			alias: certificate

The Secret must include a keystore file (keystore.jks) in JKS format as well as separate files for key (tls.key) and certificate (tls.crt). The script below shows how these can be generated and stored in the Secret. Note that this will generate a self-signed certificate, which is not suitable for use in publicly exposed interfaces. Make sure to set the parameters in the beginning of the script before using.

Example - How to generate a self-signed certificate

#!/bin/bash
KEY_PASSWORD=DefaultKeystorePWD
STORE_PASSWORD=DefaultKeystorePWD
DNAME=CN=exampledomain.com,O=Example
K8S_NAMESPACE=<namespace>

rm -f keystore.p12 keystore.jks tls.crt tls.key

keytool -genkey -keystore keystore.jks -storepass $STORE_PASSWORD -keypass $KEY_PASSWORD -alias certificate -keyalg RSA -keysize 2048 -dname $DNAME
keytool -importkeystore -srckeystore keystore.jks -srcstorepass $STORE_PASSWORD -srckeypass $KEY_PASSWORD -destkeystore keystore.p12 -deststoretype PKCS12 -srcalias certificate -deststorepass $STORE_PASSWORD -destkeypass $KEY_PASSWORD
openssl pkcs12 -in keystore.p12  -nokeys -out tls.crt -password pass:$KEY_PASSWORD
openssl pkcs12 -in keystore.p12  -nodes -nocerts -out tls.key -password pass:$KEY_PASSWORD

kubectl create secret generic mz-cert --namespace $K8S_NAMESPACE --from-file=keystore.jks --from-file=tls.key --from-file=tls.crt

Disk Based Keystore

Certificates and keys can also be stored in keystore files on persistent disk. This option is the least secure and is deprecated. It is kept solely for backward compatibility reasons.

Example - Configuration in values.yaml to enable disk based keystore

platform:
	tls:
		cert:
      		public: key
	    key:
    		alias: certificate