Container Images

Usage Engine Private Edition consists of the following container images hosted in the Digital Route AWS ECR registry:

Where <version> is the desired Usage Engine Private Edition version, for example 4.0.0.

Hosting Container Images in Your Own Container Registry

If you have your own container registry, it is recommended that you host the Usage Engine Private Edition container images there rather than in the Digital Route AWS ECR registry.

In order to access the container images in the Digital Route AWS ECR registry, you will need to authenticate yourself first. Here is how you can do this using the docker CLI:

docker login -u AWS \
-p $(AWS_ACCESS_KEY_ID=<your aws access key> AWS_SECRET_ACCESS_KEY=<your aws secret access key> aws ecr get-login-password --region eu-west-1) \
462803626708.dkr.ecr.eu-west-1.amazonaws.com

Where <your aws access key> and <your aws secret access key> are the access keys provided by Digital Route (see https://infozone.atlassian.net/wiki/spaces/UEPE4D/pages/161481605/Common+Pre-requisites#ECR-Access-Keys in case you have not received any access keys yet).

Once authenticated, you can pull the container images, re-tag them and then finally push them to your own container image repository.

Depending on how your container registry is configured, you probably need to set up an image pull secret that allows the Kubernetes cluster to pull the container images from your container registry in runtime.

Image Pull Secret for Digital Route AWS ECR

On the other hand, if you do not have your own container image registry, then you need to set up an image pull secret that allows the Kubernetes cluster to pull the container images from the Digital Route AWS ECR in runtime.

Such a secret can be created like this:

kubectl create secret docker-registry ecr-cred \
    --docker-server=https://462803626708.dkr.ecr.eu-west-1.amazonaws.com  \
    --docker-username=AWS \
    --docker-password=$(AWS_ACCESS_KEY_ID=<your aws access key> AWS_SECRET_ACCESS_KEY=<your aws secret access key> aws ecr get-login-password --region eu-west-1) \
    -n uepe

Where <your aws access key> and <your aws secret access key> are the access keys provided by Digital Route (see https://infozone.atlassian.net/wiki/spaces/UEPE4D/pages/161481605/Common+Pre-requisites#ECR-Access-Keys in case you have not received any access keys yet).

Since AWS ECR credentials expire after 12 hours, the image pull secret needs to be refreshed regularly. This can be automated through a cron job. The following yaml spec is an example of such a cron job:

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: ecr-credentials-sync
  namespace: uepe
rules:
- apiGroups: [""]
  resources:
  - secrets
  verbs:
  - get
  - create
  - patch
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: ecr-credentials-sync
  namespace: uepe
subjects:
- kind: ServiceAccount
  name: ecr-credentials-sync
roleRef:
  kind: Role
  name: ecr-credentials-sync
  apiGroup: ""
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ecr-credentials-sync
  namespace: uepe
---
apiVersion: batch/v1
kind: CronJob
metadata:
  name: ecr-credentials-sync
  namespace: uepe
spec:
  suspend: false
  schedule: 0 */8 * * *
  failedJobsHistoryLimit: 1
  successfulJobsHistoryLimit: 1
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: ecr-credentials-sync
          restartPolicy: Never
          volumes:
          - name: token
            emptyDir:
              medium: Memory
          initContainers:
          - image: amazon/aws-cli
            name: get-token
            imagePullPolicy: IfNotPresent
            env:
            - name: AWS_ACCESS_KEY_ID
              value: <your aws access key>
            - name: AWS_SECRET_ACCESS_KEY
              value: <your aws secret access key>
            - name: REGION
              value: eu-west-1
            volumeMounts:
            - mountPath: /token
              name: token
            command:
            - /bin/sh
            - -ce
            - aws ecr get-login-password --region ${REGION} > /token/ecr-token
          containers:
          - image: bitnami/kubectl
            name: create-secret
            imagePullPolicy: IfNotPresent
            env:
            - name: SECRET_NAME
              value: ecr-cred
            volumeMounts:
            - mountPath: /token
              name: token
            command:
            - /bin/sh
            - -ce
            - |-
              kubectl create secret docker-registry $SECRET_NAME \
                --dry-run=client \
                --docker-server=https://462803626708.dkr.ecr.eu-west-1.amazonaws.com \
                --docker-username=AWS \
                --docker-password="$(cat /token/ecr-token)" \
                -n uepe \
                -o yaml | kubectl apply -f -              

Where <your aws access key> and <your aws secret access key> are the access keys provided by Digital Route (see https://infozone.atlassian.net/wiki/spaces/UEPE4D/pages/161481605/Common+Pre-requisites#ECR-Access-Keys in case you have not received any access keys yet).

Simply put the above yaml spec into a file called ecr-credentials-sync.yaml, and then use the following command to create it in your Kubernetes cluster:

kubectl apply -f ecr-credentials-sync.yaml -n uepe

Cert-manager must be backed by a certificate authority (CA) to sign the certificates. Once configured with a CA, cert-manager will automatically sign and renew certificates for the system as needed. Configuring cert-manager with a CA is done by creating an Issuer or ClusterIssuer resource (this resource will be referenced later when installing Usage Engine Private Edition).

Refer to https://cert-manager.io/docs/configuration/ for a all the details.

It’s also possible to use an issuer specifiction that will issue a self-signed certificate:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: example-issuer
spec:
  selfSigned: {}

Regardless of the chosen issuer specification, to create the issuer, simply put the specification in a yaml file (here we call it example-issuer.yaml), and then execute a command like this:

kubectl apply -f example-issuer.yaml

Based on the example above the created ClusterIssuer can be inspected like this:

kubectl get clusterissuers example-issuer -o yaml

Secret

If you do not want to automate the certificate provisioning with cert-manager, you can instead manually install a public certificate in a Kubernetes Secret and then refer to that when installing Usage Engine Private Edition.

The Secret must include a keystore file (keystore.jks) in JKS format as well as separate files for key (tls.key) and certificate (tls.crt).

This is an example script that can generate a Secret like that (make sure to set the parameters at the beginning of the script before executing it):

#!/bin/sh
KEY_PASSWORD=<your chosen key password>
STORE_PASSWORD=<your chosen keystore password>
DNAME=CN=exampledomain.com,O=Example
NAMESPACE=uepe
keytool -genkey -keystore keystore.jks -storepass $STORE_PASSWORD -keypass $KEY_PASSWORD -alias certificate -keyalg RSA -keysize 4096 -dname $DNAME
keytool -importkeystore -srckeystore keystore.jks -srcstorepass $STORE_PASSWORD -srckeypass $KEY_PASSWORD -destkeystore keystore.p12 -deststoretype PKCS12 -srcalias certificate -deststorepass $STORE_PASSWORD -destkeypass $KEY_PASSWORD
openssl pkcs12 -in keystore.p12  -nokeys -out tls.crt -password pass:$KEY_PASSWORD
openssl pkcs12 -in keystore.p12  -nodes -nocerts -out tls.key -password pass:$KEY_PASSWORD
kubectl create secret generic uepe-cert -n $NAMESPACE --from-file=keystore.jks --from-file=tls.key --from-file=tls.crt

Note that this will generate a self-signed certificate, which is not suitable for use in publicly exposed interfaces.

Once the Secret has been generated, its content can be inspected like this:

kubectl -n uepe get secrets uepe-cert -o yaml

Configuring Kubernetes Secrets for System Credentials

Anchor
bootstrapping-system-credentials bootstrapping-system-credentials

Usage Engine Private Edition uses a number of system credentials in order to function as expected.

These system credentials are kept in a Kubernetes secret called env-secrets located in the same namespace as where Usage Engine Private Edition is being installed.

This secret can be populated in three different ways:

• Manually creating and populating it prior to installing Usage Engine Private Edition.

• Providing the credential(s) as helm values at install time. In which case the secret will be automatically created (if it does not already exist) and populated with the corresponding helm value(s). Be aware that storing credentials in a values.yaml file in version control is not secure. If you still need to do this you should consider using tools like https://github.com/mozilla/sops .

• Letting it be automatically populated at install time. In which case the secret will be automatically created and populated. Passwords will consist of eight randomly generated characters.