Set Up Kubernetes Cluster - OCI (4.3)

Owned by Jenni Wallin
Last updated: Dec 17, 2024 by Kam Heng Choy
7 min read

Before installing Usage Engine Private Edition, you need to set up a Kubernetes cluster on OCI OKE (Oracle’s managed Kubernetes service).

First, you need to create a basic Kubernetes cluster. You can do this in two different ways:

  • Using the terraform tool.

  • Using the OCI management console.

In this guide, terraform will be used, mainly because it will enable you to create the basic Kubernetes cluster in minutes with just a single command.

Once the basic Kubernetes cluster has been created, you need to add additional infrastructure. You can use terraform for this as well.

Before proceeding, go to Release Information, and download the oci.tar.gz file for the Usage Engine Private Edition version that you want to install. Once downloaded, extract its content to a suitable location.

Assumptions

There are a few assumptions been made when using terraform to create cluster resources:

  1. We assume that you have an existing parent domain, in the example below example.com, hosted on the same account as the cluster that we are going to create in the following section and that you want to access the cluster environment via the hostname. Terraform will create a subdomain in the following format: <cluster_name>.<domain>.

    1. cluster name: uepe-oke

    2. domain: example.com

    3. final domain: uepe-oke.example.com

  2. We also assume that terraform is allowed to add a NS (NameServer) record to the parent domain which is needed to allow DNS delegation from the parent domain to subdomain.

  3. Terraform needs to persist the state of your provisioned infrastructure. By default, the state file is stored locally on the computer that terraform is executed from. However, if multiple persons are working on the infrastructure, then it is recommended to store the state file using a remote persistence such as Object Storage, see Using Object Storage for State Files for more information.

  4. The OCI File System service (NFS) is used as the default persistent storage for data that needs to be persisted.

  5. The OCI Managed PostgreSQL service is used as the Usage Engine Private Edition database.

  6. The user Principle is used throughout the entire installation. The user must prepare the private key file locally. The user can create and download the private key via the OCI console by selecting Profile | My Profile | API keys | Add API key.

Create Basic Cluster and additional infrastructure

To create a basic Kubernetes cluster with public and private VPC:

  1. Go to <folder where you extracted the oci.tar.gz file>/oci/terraform and copy theterraform.tfvars.example to terraform.tfvars.

  2. Edit the terraform.tfvars file.

  3. Specify the desired cluster name, OCI region and kubernetes_version (see Compatibility Matrix (4.3) to find out which Kubernetes versions that are compatible with this release of Usage Engine Private Edition). Specify your OCI tenancy_ocid, user_ocid, fingerprint, compartment_ocid and private_key_path (which can be found on the OCI dashboard’s Profile page), as well as the desired number of nodes per cluster (oke_num_nodes).

  4. If you are going to use another database than Derby, specify db_password, db_version and db_username.

terraform.tfvars

Where to get the value from?

terraform.tfvars

Where to get the value from?

tenancy_ocid

In the OCI management console, the tenancy_ocid is listed on Profile | Tenancy: <tenant-name> | Tenancy Details.

fingerprint

Fingerprint is only available when the user has created API keys, see private_key_path below.

In the OCI management console, the fingerprint is listed on Profile | My Profile | Resources | API keys when the API keys have been created.

user_ocid

In the OCI management console, user_ocid is listed on Profile | My Profile

private_key_path

The full path to your private key file’s filename.

To create and download your private key, go to Profile | My Profile | Resources | API keys, create your API key and click Download.

region

The region in which you will install your cluster, for example "eu-frankfurt-1".

cluster_name

A name for your cluster. Cluster names must start with a lowercase letter followed by up to 39 lowercase letters, numbers or hyphens. They cannot end with a hyphen. The cluster name must be unique in the project.

domain

Your existing domain name. In the OCI management console, this is the DNS name that is listed on Networking |DNS management | Zones.

The service hostname created by Usage Engine Private Edition will be accessible in the following format: <service_name>.<cluster_name>.<domain> i.e. desktop-online.uepe-oke.example.com.

kubernetes_version

The kubernetes version in alpha numeric string, for example “v1.29.1".

oke_num_nodes

The number of cluster nodes in numeric format, for example “3”.

oke_availability_domain

The availability domain name for the cluster, for example "Vafx:EU-FRANKFURT-1-AD-1".

db_password

Choose a secure password for the system database administrator, minimum 10 characters.

db_version

The database version in numeric format, for example “14“.

oke_image_id

The OCID of the image to be used for worker node instance creation.

To see the available image under your compartment, use the command:

oci ce node-pool-options get --node-pool-option-id all --compartment-id <your compartment ocid>.

db_enabled

db_enables is a boolean flag for enabling cloud SQL database resource creation.

fss_enabled

fss_enabled is a boolean flag for enabling file storage resource creation. It is set to false by default. Set it to true if you need persistent file storage.

auto_create_ns_record

auto_create_ns_record is a boolean flag for enabling subdomain NS record to be automatically created in the parent domain. If your parent domain is not under the same compartment, or if your parent domain is hosted in another cloud provider, then you must set it to false.

Example

# ____ _____ _____ _____ _ _ _____ ____ _____ # / ___|| ____|_ _| |_ _| | | | ____/ ___|| ____|_ # \___ \| _| | | | | | |_| | _| \___ \| _| (_) # ___) | |___ | | | | | _ | |___ ___) | |___ _ # |____/|_____| |_| |_| |_| |_|_____|____/|_____(_) # The below values must be set explicitly in order for the setup to work correctly. tenancy_ocid = "ocid1.tenancy.oc1..aaaaaaaamnl7f7t2yrlas2si7b5hpo6t23dqi6mjo3eot6ijl2nqcog5h6ha" fingerprint = "7d:67:b3:9d:a3:8f:6d:37:f3:e9:7d:e5:45:ec:df:56" user_ocid = "ocid1.user.oc1..aaaaaaaauhk3uhiryg7sw2xjmvf45zasduqwr2cium53gmdxwipe4iqdrfuq" private_key_path = "/Users/kamheng.choy/Downloads/kamheng.choy@digitalroute.com_2024-04-07T10_07_56.490Z.pem" # Deployment compartment compartment_ocid = "ocid1.compartment.oc1..aaaaaaaa56wmblidgvvicamsqkf7sqcqu5yxdhvu3wlvomzgonhflcrv6kcq" # region region = "eu-frankfurt-1" # Name of the cluster, it must be unique in the project. cluster_name = "test-uepe-cluster-1" # Domain DNS name # We'll create a subdomain zone from parent domain, the final domain will be in format "<cluster_name>.<domain>". # Please note that if this domain is hosted on another OCI project or other cloud provider, then you must # set auto_create_ns_record = false and manually add the subdomain NS record to the parent domain. # auto_create_ns_record = false domain = "stratus.oci.digitalroute.net" # Admin user password to the database db_password = "super_SeCrEt_db_pAsSwOrD_457!" # _______ _______ _ _ __ _ ____ _ _____ # |_ _\ \ / / ____| / \ | |/ / / \ | __ )| | | ____|_ # | | \ \ /\ / /| _| / _ \ | ' / / _ \ | _ \| | | _| (_) # | | \ V V / | |___ / ___ \| . \ / ___ \| |_) | |___| |___ _ # |_| \_/\_/ |_____/_/ \_\_|\_\/_/ \_\____/|_____|_____(_) # The below sections are the default values, tweak them to your needs. # Kubernetes version kubernetes_version = "v1.29.1" # Number of nodes per cluster oke_num_nodes = 3 # Worker node machine type node_pool_shape = "VM.Standard.E4.Flex" oke_availability_domain = "Vafx:EU-FRANKFURT-1-AD-1" oke_image_id = "ocid1.image.oc1.eu-frankfurt-1.aaaaaaaapwbqurbd2hpmj2at354r3dkok4o4644am4hwgdagoekpcaon7shq" # IP CIDR range allocate to the control plane vcn_cidr_blocks = "10.0.0.0/16" # Network file system (NFS) persistent storage fss_enabled = true fss_availability_domain = "Vafx:EU-FRANKFURT-1-AD-1" # Cloud SQL database db_enabled = true # DB instance type db_instance_shape = "PostgreSQL.VM.Standard.E4.Flex.4.64GB" # DB version db_version = "14"

 

Note!

If your parent domain zone is not under the same project:

  • Set auto_create_ns_record = false to disable subdomain NS record auto creation in the parent domain.

  • Perform terraform apply.

  • When terraform has been applied, copy the name server's value from the terraform output and manually add them to parent domain as a NS record. If you are not using OCI DNS as the parent domain, see your Domain Registrar documentation for information on how to add NS record.

  1. Run the following commands:

terraform init terraform plan terraform apply

  1. Wait until the terraform commands have completed and you see the following kind of information:

Ensure to save the output from terraform above since it will be used as input throughout the remainder of this installation guide.

A basic Kubernetes cluster has now been set up successfully.

An RDS PostgreSQL database instance is up and running on a private subnet VPC with default listening port 5432. The default database postgres is accessible within the cluster at end point db5j5pt3qwjqmmjgfremgugr7cxtsq-dbinstance-70c946d1330e.postgresql.eu-frankfurt-1.oc1.oraclecloud.com with admin username postgres.

You can see the status of the cluster, db and the other resources in the OCI dashboard.

Configure Cluster Access

To configure cluster access, run the following command:

A ./kubeconfig.yaml file containing information on how to connect to your newly created cluster will be generated. Set the KUBECONFIG environment variable to point to that file by running the following command:

This will ensure that tools like kubectl and helm will connect to your newly created cluster.

You can check the status of the cluster nodes by running the following command:

In this example cluster, the output will look something like this:

 

This section is now complete and you can proceed to the Kubernetes Cluster Add-ons - OCI (4.3) section.