OCI Add-ons

Add the following OCI-specific resources:

• oci-file-service-storage

• oci-native-ingress-controller

oci-file-service-storage

The OCI File Storage service provides a durable, scalable, distributed, and enterprise-grade network file system.

A persistent volume claim (PVC) is a request for persistent file storage. The OCI File Storage service file systems are mounted inside containers running on clusters created by Container Engine for Kubernetes using a CSI (Container Storage Interface) volume plugin deployed on the clusters.

To enable the CSI volume plugin to create and manage File Storage resources, the appropriate IAM policies must be applied by following these steps:

1. Apply policy to create and/or manage file systems, mount targets, and export paths:

2. Apply policy to use VNICs, private IPs, private DNS zones, and subnets:

Update Default CSI Driver

When a pod attempts to access a persistent volume (PV) backed by a file system in the File Storage service, the attempt can fail with a "Permission Denied" message since the volume is only accessible to processes running as root. As a result, a pod that is not running as root receives the "Permission Denied" message when attempting to access a directory or file in the mounted volume.

To avoid getting the "Permission Denied” message, follow these steps:

1. Obtain the CSIDriver configuration file by running the following command:

2. Edit the fss_csi_driver.yaml file and change the CSIDriver object's spec.fsGroupPolicy attribute from ReadWriteOnceWithFSType to File, for example as below:

3. Delete the existing CSIDriver object by running the following command:

4. Create the new CSIDriver object from fss_csi_driver.yaml by running the following command:

For more information, see Troubleshooting File Storage Service Provisioning of PVCs.

Provisioning of PVC

You can use the File Storage service to provision persistent volume claims (PVCs) in two ways:

• Dynamic Provisioning (deprecated way)

• Static Provisioning (preferred way)

Static Provisioning

Follow these steps to create a PVC by creating a PV backed by the new file system and then create the PVC and binds the PVC to the PV backed by the File Storage service:

1. Prepare a pv.yaml file with PersistentVolume manifest for OCI File Storage with the following content:

2. Deploy the PersistentVolume by running the following command:

3. Prepare a pvc.yaml file with PersistentVolumeClaim manifest for OCI File Storage with the following content:

4. Deploy the PersistentVolumeClaim by running the following command:

5. Verify that PVC is bound to the PV successfully by running the following command:

The output below shows that a persistent volume claim is successfully bound to a persistent volume.

oci-native-ingress-controller

The OCI native ingress controller implements the rules and configuration options defined in a Kubernetes ingress resource to load balance and route incoming traffic to service pods running on worker nodes in a cluster. The OCI native ingress controller creates an OCI flexible load balancer to handle requests, and configures the OCI load balancer to route requests according to the rules defined in the ingress resource.

The OCI Native Ingress controller creates the following OCI load balancer resources:

• A load balancer for each IngressClass resource where you have specified the OCI native ingress controller as the controller.

• A load balancer backend set for each unique Kubernetes service name and port number combination that you include in routing rules in Ingress resources in the cluster.

• A routing policy that reflect the rules defined in the ingress resource that is used to route traffic to backend set.

• A load balancer listener for each unique port that you include in routing rules in Ingress resources in the cluster.

To install OCI Native Ingress Controller:

1. Create a config file named user-auth-config.yaml, containing credential information, in the following format:

2. Create a Kubernetes secret resource named oci-config in the cluster by running the following command:

3. Grant permission to the OCI Native Ingress Controller to access resources created by other OCI services, such as the Load Balancer service and the Certificates service by installing these IAM policies:

4. Clone the OCI native ingress controller repository from GitHub by running the following command:

5. In the local Git repository, navigate to the oci-native-ingress-controller directory and create a config file named oci-native-ingress-controller-values.yaml with the following content:

6. Install the config file oci-native-ingress-controller-values.yaml by running the following command:

7. Confirm that the OCI native ingress controller has been installed successfully by running the following command:

The logs should look similar to:

When you have installed the OCI native ingress controller, you must created the following Kubernetes resources in order to start using it.

• IngressClassParameters

• IngressClass

IngressClassParameters resource

Use the custom IngressClassParameters resource to specify the details of the OCI load balancer you create for the OCI native ingress controller.

Define the resource in a .yaml file named ingress-class-params.yaml as in the example below:

To create the resource, run the following command:

IngressClass resource

Use the IngressClass resource to associate an Ingress resource with the OCI native ingress controller and the IngressClassParameters resource.

Define the resource in a .yaml file named ingress-class.yaml as in the example below:

To create the resource, run the following command:

Kubernetes Add-ons

Add the following general Kubernetes resources:

• external-dns

• ingress-nginx-controller

external-dns

ExternalDNS is a Kubernetes add-on that configures public DNS servers with information about exposed Kubernetes services to make them discoverable.

To install ExternalDNS:

1. Create a Kubernetes secret containing the Oracle Cloud Infrastructure user authentication details for that the ExternalDNS can use when connecting to the Oracle Cloud Infrastructure API for inserting and updating DNS records in the DNS zone. Create a credentials file named oci.yaml with the following content:

2. Create a Kubernetes secret named external-dns-config from the credentials file you just created by running the following command:

3. Create a configuration file (for example, called external-dns-values.yaml), and specify the name of the Kubernetes secret you just created as in the example below:

4. Add the bitnami helm repository by running the following command:

5. Update the helm repository to get the latest software by running the following command:

6. Do a helm install with the yaml file external-dns-values.yaml to deploy ExternalDNS:

5. Confirm that external-dns has been installed successfully by running the following command:

The logs should look similar to the example below:

ingress-nginx-controller

The Ingress NGINX Controller is an ingress controller for Kubernetes using NGINX as a reverse proxy and load balancer.

To install the Ingress NGINX Controller, follow these steps:

1. Add the ingress-nginx helm repository:

2. Update the helm repository to get the latest software:

3. Create a file called ingress-nginx-values.yaml and populate it with the following helm values:

4. Install the ingress-nginx-controller helm chart:

   Where <helm chart version> is a compatible version listed in the Compatibility Matrix (4.2).

If you run the helm list -A command you will see all add-ons added in this section, for example like below:

This section is now complete and you can proceed to the Usage Engine Private Edition Preparations - OCI (4.2) section.