Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
languagetext
#  ____  _____ _____   _____ _   _ _____ ____  _____
# / ___|| ____|_   _| |_   _| | | | ____/ ___|| ____|_
# \___ \|  _|   | |     | | | |_| |  _| \___ \|  _| (_)
#  ___) | |___  | |     | | |  _  | |___ ___) | |___ _
# |____/|_____| |_|     |_| |_| |_|_____|____/|_____(_)

# The below values must be set explicitly in order for the setup to work correctly.

# Project settings, use command `gcloud projects list` to retrieve project info.
project_id = "pt-dev-stratus-bliz"
project_number = "413241157368"

# Region to deploy, use command `gcloud compute regions list` to get available regions.
region = "europe-north1"

# Name of the cluster, it must be unique in the project.
cluster_name = "my-uepe-gketenancy_ocid     = "ocid1.tenancy.oc1..aaaaaaaamnl7f7t2yrlas2si7b5hpo6t23dqi6mjo3eot6ijl2nqcog5h6ha"
fingerprint      = "7d:67:b3:9d:a3:8f:6d:37:f3:e9:7d:e5:45:ec:df:56"
user_ocid        = "ocid1.user.oc1..aaaaaaaauhk3uhiryg7sw2xjmvf45zasduqwr2cium53gmdxwipe4iqdrfuq"
private_key_path = "/Users/kamheng.choy/Downloads/kamheng.choy@digitalroute.com_2024-04-07T10_07_56.490Z.pem"

# Deployment compartment
compartment_ocid = "ocid1.compartment.oc1..aaaaaaaa56wmblidgvvicamsqkf7sqcqu5yxdhvu3wlvomzgonhflcrv6kcq"

# region
region = "eu-frankfurt-1"

# DomainName DNSof namethe # The DNS zone cluster, it must alreadybe existunique in the Cloud DNS or in other cloud provider DNS zone.project.
cluster_name = "test-uepe-cluster-1"

# Domain DNS name
# We'll create a subdomain zone from parent domain, the final domain will be in format "<cluster_name>.<domain>".
# Please note that if this domain is hosted on another GCP project or other cloud provider, then you must
# set auto_create_ns_record = false and manually add the subdomain NS record to the parent domain.
domain = "pe-mzstratus.gcpoci.digitalroute.net"

# Admin user password to the database
db_password = "super_SeCrEt_db_pAsSwOrD_457"

.........

#"Password123$"

#  _______        _____ _ _    _    _   __ # |  ___(_) |     ____  ___| |_ ___  _ __ ___
# |_ |_  _\ \    | | |/ _ \/ ____| __ / \  _| \|/ '__/   _/ \ # | __ _)| |  | |  ___/\_|_
\# || (_) | | | \ __\ /\ #/ /|  _|   / |_|_|\___||___/\__\___/|_|  \___|

# Network file system (NFS) persistent storage
# For testing purpose, you could use block storage as alternative cheaper option.
# However do note that block storage has its limitation where it only works for single node cluster setup (ReadWriteOnce access mode).
# See https://cloud.google.com/kubernetes-engine/docs/concepts/storage-overview for explanation.
filestore_enabled = true
# Service tier of the instance
# See https://cloud.google.com/filestore/docs/reference/rest/v1/Tier for available service tier.
filestore_service_tier = "STANDARD"
# Location of the instance, you MUST set a zone if the service tier is not ENTERPRISE. For ENTERPRISE tier, this can be a region.
# To find out available zones of your region, use command `gcloud compute zones list --filter="region:europe-north1"`.
filestore_location = "europe-north1-a"
# Storage capacity in GB, must be at least 1024
filestore_capacity = 1024
# The name of the fileshare (16 characters or less)
fileshare_name = "share1"

Important notes if your parent domain zone is not under the same project:

...

 \ | ' /   / _ \ |  _ \| |   |  _| (_)
#   | |   \ V  V / | |___ / ___ \| . \  / ___ \| |_) | |___| |___ _
#   |_|    \_/\_/  |_____/_/   \_\_|\_\/_/   \_\____/|_____|_____(_)

# The below sections are the default values, tweak them to your needs.

# Kubernetes version
kubernetes_version = "v1.29.1"

# Number of nodes per cluster
oke_num_nodes = 3
# Worker node machine type
node_pool_shape = "VM.Standard.E4.Flex"
oke_availability_domain = "Vafx:EU-FRANKFURT-1-AD-1"

oke_image_id = "ocid1.image.oc1.eu-frankfurt-1.aaaaaaaapwbqurbd2hpmj2at354r3dkok4o4644am4hwgdagoekpcaon7shq"

# IP CIDR range allocate to the control plane
vcn_cidr_blocks = "10.0.0.0/16"

Important notes if your parent domain zone is not under the same project:

  • You need to set auto_create_ns_record = false to disable subdomain NS record auto creation in the parent domain.

  • Perform terraform apply.

  • After terraform apply is finished, copy the name servers value from terraform output and manually add them to parent domain as NS record. If you are not using OCI DNS as the parent domain, please refer to your Domain Registrar documentation on how to add NS record.

...

Code Block
languagetext
Apply complete! Resources: 20 added, 0 changed, 0 destroyed.

Outputs:

cert_manager_namespace = "cert-manager"
cert_manager_service_account = "cert-manager-my-uepe-gke-1@pt-dev-stratus-bliz.iam.gserviceaccount.com"
db_endpoint = "db.my-uepe-gke-1.pe-mz.gcp.digitalroute.net"
external_dns_namespace = "uepe"
external_dns_service_account = "external-dns-my-uepe-gke-1@pt-dev-stratus-bliz.iam.gserviceaccount.com"
filestore_capacity_gb = 1024
filestore_csi_volume_handle = "modeInstance/europe-north1-a/my-uepe-gke-1-filestore/share1"
filestore_ip_address = "10.143.245.42"
filestore_persistence_yaml = "./manifests/filestore_persistence.yaml"
filestore_share_name = "share1"
gke_domain_dns_name = "my-uepe-gke-1.pe-mz.gcp.digitalroute.net"
gke_domain_zone_name = "my-uepe-gke-1-pe-mz-gcp-digitalroute-net"
kubernetes_cluster_host = "34.124.151.111"
kubernetes_cluster_location = "europe-north1"
kubernetes_cluster_name = "my-uepe-gke-1"
name_servers = tolist([
  "ns-cloud-b1.googledomains.com.",
  "ns-cloud-b2.googledomains.com.",
  "ns-cloud-b3.googledomains.com.",
  "ns-cloud-b4.googledomains.com.",
])
project_id = "pt-dev-stratus-bliz"
project_number = "413241157368"
region = "europe-north1"
Info

Make sure to save the output from terraform above. Reason being that it is used as input throughout the remainder of this installation guide.

A fully functional Kubernetes cluster has now been set up successfully.

A RDS PostgreSQL database instance up and running on private subnet VPC with default listening port 5432. The default database postgres is accessible within the cluster at end point db.my-uepe-gke-1.pe-mz.gcp.digitalroute.net with admin username postgres.

You can check the status of the cluster, db and the other resources in the OCI dashboard.

Setup Additional Infrastructure Resources on AWS

At this stage, a basic Kubernetes cluster has been created. However, some additional infrastructure resources remain to be set up. Namely the following:

  • Hosted Zone (subdomain) for domain name.

  • ACM Certificate for the domain name (to be used with any load balancers).

  • KMS CMK key which is used for encryption at-rest for EFS, RDS and SSM.

  • EFS with security group in place.

  • RDS PostgreSQL with security group in place.

Follow these steps to set up the remaining infrastructure resources:

  1. Go to <the location where you extracted the aws.tar.gz file>/terraform

  2. Copy terraform.tf.vars.example to terraform.tfvars.

  3. Retrieve the following values from AWS Console and fill in the parameters in terraform.tfvars

...

terraform.tfvars

...

Where to get the value from?

...

vpc_id

...

In the AWS management console, you can find this information by searching for “Your VPCs”. Pick the VPC ID of the cluster that you created in the previous section.

...

aws_region

...

From metadata.region in your uepe-eks.yamlfile.

...

aws_account_id

...

In the AWS management console, this is the Account ID that is listed on your Account page.

...

cluster_name

...

From metadata.name in your uepe-eks.yaml file.

...

domain

...

In the AWS management console, on the Route 53 service page, this is the Hosted zone name of your existing Hosted zone.

...

domain_zone_id

...

In the AWS management console, on the Route 53 service page, this is the Hosted zone ID of your existing Hosted zone.

...

db_password

...

Choose a secure password for the system database administrator.

Minimum 10 characters.

Example:

Code Block
#  ____  _____ _____   _____ _   _ _____ ____  _____
# / ___|| ____|_   _| |_   _| | | | ____/ ___|| ____|_
# \___ \|  _|   | |     | | | |_| |  _| \___ \|  _| (_)
#  ___) | |___  | |     | | |  _  | |___ ___) | |___ _
# |____/|_____| |_|     |_| |_| |_|_____|____/|_____(_)

# The below values must be set explicitly in order for the setup to work correctly.

vpc_id = "vpc-04ff16421e3ccdd94"
aws_region = "eu-west-1"
aws_account_id = "058264429588"

# Name of the cluster, it must be unique in the account.
cluster_name = "example-cluster"

# Domain DNS name
# The DNS zone must already exist in Route53 or in other cloud provider DNS zone.
# We'll create a subdomain zone from parent domain, the final domain will be in format "<cluster_name>.<domain>".
# Please note that if this domain is hosted on another AWS account or other cloud provider, then you must
# set auto_create_ns_record = false and manually add the subdomain NS record to the parent domain.
domain = "stratus.digitalroute.net"

# Admin user password to the database.
db_password = "super_SeCrEt_db_pAsSwOrD_457!"
Info

Important notes if your parent domain zone is not under the same account:

  • You need to set auto_create_ns_record = false to disable subdomain NS record auto creation in the parent domain.

  • Terraform apply will fail due to certificate validation timeout error │ Error: waiting for ACM Certificate (arn:aws:acm:ap-southeast-1:027763730008:certificate/84ae1022-15bd-430a-ab3e-278f01b0edb6) to be issued: timeout while waiting for state to become 'ISSUED' (last state: 'PENDING_VALIDATION', timeout: 2m0s)

  • When the error above happened, you need to manually retrieve the name servers value from the created subdomain and add them to parent domain as NS record. If you are not using Route53 as the parent domain, please refer to your Domain Registrar documentation on how to add NS record.

  • Once NS record is added to the parent domain, go to AWS Console |  AWS Certificate Manager (ACM) and wait for the certificate status become verified. It will take 10-20 minutes.

  • After the certificate is verified, run the terraform apply again to continue provisioning.

  1. Run the following commands

Code Block
languagebash
terraform init
terraform plan
terraform apply
  1. Wait for the terraform commands to finish.

Code Block
languagebash
Apply complete! Resources: 16 added, 0 changed, 0 destroyed.

Outputs:

certificate_arn = "arn:aws:acm:eu-west-1:058264429588:certificate/526ed179-afa7-4778-b1b8-bfbcb95e4534"
db_endpoint = "example-cluster-db.c70g0ggo8m66.eu-west-1.rds.amazonaws.com:5432"
db_password = <sensitive>
db_user = "dbadmin"
efs_id = "fs-0f0bb5c0ef98f5b6f"
eks_domain_zone_id = "Z076760737OMHF392P9P7"
eks_domain_zone_name = "example-cluster.stratus.digitalroute.net
db_endpoint = "db.my-uepe-gke-1.pe-mz.gcp.digitalroute.net"
external_dns_namespace = "uepe"
external_dns_service_account = "external-dns-my-uepe-gke-1@pt-dev-stratus-bliz.iam.gserviceaccount.com"
filestore_capacity_gb = 1024
filestore_csi_volume_handle = "modeInstance/europe-north1-a/my-uepe-gke-1-filestore/share1"
filestore_ip_address = "10.143.245.42"
filestore_persistence_yaml = "./manifests/filestore_persistence.yaml"
filestore_share_name = "share1"
gke_domain_dns_name = "my-uepe-gke-1.pe-mz.gcp.digitalroute.net"
gke_domain_zone_name = "my-uepe-gke-1-pe-mz-gcp-digitalroute-net"
kubernetes_cluster_host = "34.124.151.111"
kubernetes_cluster_location = "europe-north1"
kubernetes_cluster_name = "my-uepe-gke-1"
name_servers = tolist([
  "ns-1344.awsdnscloud-40.org",
  "ns-2018.awsdns-60.co.ukb1.googledomains.com.",
  "ns-55.awsdns-06cloud-b2.googledomains.com.",
  "ns-cloud-664b3.awsdns-19.net",
])
private_subnets = [
  "subnet-0956aa9898f78900dgoogledomains.com.",
  "subnet-0b6d1364dfb4090d6",
  "subnet-0da06b6a88f9f45e7ns-cloud-b4.googledomains.com.",
])
publicproject_subnetsid = [   "subnet-01174b6e86367827b",
  "subnet-0d0b14a68fe42ba09",
  "subnet-0eed6adde0748e1f6",
]"pt-dev-stratus-bliz"
project_number = "413241157368"
region = "europe-north1"
Info

Make sure to save the output from terraform above. Reason being that it is used as input throughout the remainder of this installation guide.

A basic Kubernetes cluster has now been createdset up successfully.

A RDS PostgreSQL database instance up and running on private subnet VPC with default listening port 5432. The default database PlatformDatabase postgres is accessible within the cluster at end point example-cluster-db.c70g0ggo8m66.eu-westmy-uepe-gke-1.pe-mz.rdsgcp.amazonawsdigitalroute.comnet with admin username dbadmin. postgres.

You can check the status of the cluster, db and the other resources in the OCI dashboard.

Create Namespace

Code Block
languagebash
kubectl create namespace uepe 

Now proceed to the Kubernetes Cluster Add-ons - OCI section.