Kubernetes Cluster Add-ons - OCI

OCI Add-ons

The following AWS specific resources should be added:

• efs-csi-controller

• oci-native-ingress-controller

efs-csi-controller

The Amazon Elastic File System Container Storage Interface (CSI) Driver implements the CSI specification for container orchestrators to manage the lifecycle of Amazon EFS file systems.

To install the Amazon EFS CSI Driver, follow these steps:

1. Add the Amazon EFS CSI Driver helm repository:

helm repo add aws-efs-csi-driver https://kubernetes-sigs.github.io/aws-efs-csi-driver/

2. Update helm repository to get the latest software:

helm repo update

3. Helm install:

helm install aws-efs-csi-driver --namespace uepe aws-efs-csi-driver/aws-efs-csi-driver --version <helm chart version> \
--set controller.serviceAccount.create=false \
--set controller.serviceAccount.name=efs-csi-controller-sa

Where <helm chart version> is a compatible version listed in the Compatibility Matrix.

Dynamic Provisioning

These steps describe how to create a dynamically provisioned volume created through Amazon EFS access points and a corresponding persistent volume claim (PVC).

1. Prepare a storageclass.yaml file with StorageClass manifest for Amazon EFS:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: aws-efs
provisioner: efs.csi.aws.com
parameters:
  provisioningMode: efs-ap
  fileSystemId: <efs_id from terraform output>
  directoryPerms: "700"

2. Deploy the storage class

kubectl apply -f storageclass.yaml

For more information, please refer to the dynamic provisioning documentation.

oci-native-ingress-controller

The OCI Native Ingress controller manages the following OCI resources:

• Application Load Balancers to satisfy Kubernetes Ingress resources.

• Network Load Balancers to satisfy Kubernetes Service resources of type LoadBalancer with appropriate annotations.

To install AWS Load Balancer Controller, follow these steps:

1. Add eks repository to the helm repository:

helm repo add eks https://aws.github.io/eks-charts

2. Update helm repository to get the latest software:

helm repo update

3. Install the AWS Load Balancer Controller helm chart:

helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
-n uepe --version <helm chart version> \
--set clusterName=<cluster_name configured in terraform.tfvars> \
--set serviceAccount.create=false \
--set serviceAccount.name=aws-load-balancer-controller

Where <helm chart version> is a compatible version listed in the Compatibility Matrix.

Kubernetes Add-ons

The following general Kubernetes resources should be added:

• external-dns

• ingress-nginx-controller

external-dns

ExternalDNS is a Kubernetes add-on that configures public DNS servers with information about exposed Kubernetes services to make them discoverable.

To install ExternalDNS, follow these steps:

1. Add the bitnami helm repository:

   helm repo add bitnami https://charts.bitnami.com/bitnami

2. Update the helm repository to get the latest software:

   helm repo update

3. Create a file called external-dns-values.yaml and populate it with the following helm values:

   aws:
     zoneType: public
   domainFilters:
     - <eks_domain_zone_name from terraform output>
   policy: sync
   provider: aws
   txtOwnerId: <eks_domain_zone_id from terraform output>
   serviceAccount:
     create: false
     name: external-dns

4. Install the ExternalDNS helm chart:

   helm install external-dns bitnami/external-dns -n uepe \
   --version <helm chart version> -f external-dns-values.yaml

   Where <helm chart version> is a compatible version listed in the Compatibility Matrix.

ingress-nginx-controller

The Ingress NGINX Controller is an ingress controller for Kubernetes using NGINX as a reverse proxy and load balancer.

To install the Ingress NGINX Controller, follow these steps:

1. Add the ingress-nginx helm repository:

   helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx

2. Update the helm repository to get the latest software:

   helm repo update

3. Create a file called ingress-nginx-values.yaml and populate it with the following helm values:

   controller:
     scope:
       enabled: true
     admissionWebhooks:
       enabled: false
     metrics:
       enabled: false
       serviceMonitor:
         enabled: false
     ingressClassResource:
       name: nginx
       enabled: true
       default: false
       controllerValue: "k8s.io/ingress-nginx"
     watchIngressWithoutClass: false
     service:
       targetPorts:
         http: 80
         https: 443
       type: NodePort
     extraArgs:
       v: 1
   serviceAccount:
     create: false

4. Install the ingress-nginx-controller helm chart:

   helm install ingress-nginx ingress-nginx/ingress-nginx --version <helm chart version> -f ingress-nginx-values.yaml -n uepe

   Where <helm chart version> is a compatible version listed in the Compatibility Matrix.

Executing helm list should show all add-ons added in this section. Example:

NAME                        	NAMESPACE	REVISION	UPDATED                             	STATUS  	CHART                             	APP VERSION
aws-efs-csi-driver          	uepe     	1       	2024-02-06 14:00:36.817518 +0800 +08	deployed	aws-efs-csi-driver-2.5.4          	1.7.4      
aws-load-balancer-controller	uepe     	1       	2024-02-06 14:09:22.86071 +0800 +08 	deployed	aws-load-balancer-controller-1.7.0	v2.7.0     
external-dns                	uepe     	1       	2024-02-06 14:06:28.705309 +0800 +08	deployed	external-dns-6.31.5               	0.14.0     
ingress-nginx-controller        uepe     	1       	2024-02-22 11:44:54.18561 +0800 +08 	deployed	ingress-nginx-4.9.1               	1.9.6

This section is now complete. Now proceed to the Usage Engine Private Edition Preparations - OCI section.