{"type":"doc","content":[{"type":"paragraph","content":[{"text":"Before installing Usage Engine Private Edition, you must set up a Kubernetes cluster on Azure AKS (Azure’s managed Kubernetes service).","type":"text"}]},{"type":"paragraph","content":[{"text":"First, a basic Kubernetes cluster needs to be created. This can be done in two different ways:","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Using the ","type":"text"},{"text":"terraform","type":"text","marks":[{"type":"code"}]},{"text":" tool.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Using the Azure management console.","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"In this guide, ","type":"text"},{"text":"terraform","type":"text","marks":[{"type":"code"}]},{"text":" will be used. Mainly because it will enable you to create the basic Kubernetes cluster in minutes with just a single command.","type":"text"}]},{"type":"paragraph","content":[{"text":"Once the basic Kubernetes cluster has been created, additional infrastructure must be added. For this ","type":"text"},{"text":"terraform","type":"text","marks":[{"type":"code"}]},{"text":" is also used.","type":"text"}]},{"type":"paragraph","content":[{"text":"Before proceeding, go to ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://infozone.atlassian.net/wiki/spaces/UEPE4D/pages/107381361"}},{"text":", and download the ","type":"text"},{"text":"azure.tar.gz","type":"text","marks":[{"type":"code"}]},{"text":" file for the Usage Engine Private Edition version that is being installed. Once downloaded, extract its content to a suitable location.","type":"text"}]},{"type":"paragraph","content":[{"text":"Assumptions","type":"text","marks":[{"type":"strong"}]}]},{"type":"paragraph","content":[{"text":"There are a few assumptions which have been made when using terraform to create cluster resources:","type":"text"}]},{"type":"orderedList","attrs":{"order":1},"content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"We assume you have an existing parent domain i.e. ","type":"text"},{"text":"example.com","type":"text","marks":[{"type":"strong"}]},{"text":" hosted on the ","type":"text"},{"text":"same","type":"text","marks":[{"type":"strong"}]},{"text":" account as the cluster that we are going to create in the coming section and you wish to access the cluster environment through the hostname. Terraform will create a subdomain in format ","type":"text"},{"text":".","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"}]},{"type":"orderedList","attrs":{"order":1},"content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"cluster name: uepe-aks","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"domain: example.com","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"final domain: ","type":"text"},{"text":"uepe-aks.example.com","type":"text","marks":[{"type":"strong"}]}]}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"In addition, we also assume terraform is allowed to add a NS (NameServer) record to the parent domain. This is to allow ","type":"text"},{"text":"DNS delegation","type":"text","marks":[{"type":"link","attrs":{"href":"https://www.cloudns.net/blog/dns-delegation/"}}]},{"text":" from the parent domain to subdomain.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Terraform needs to persist the state of your provisioned infrastructure, by default the state file is stored locally on the computer that terraform is executed from. However if you have multiple person working on the infrastructure then it is recommended to store the state file on remote persistent such as Azure Storage, see ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://learn.microsoft.com/en-us/azure/developer/terraform/store-state-in-azure-storage?tabs=terraform"}},{"text":" for more information.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"We use the ","type":"text"},{"text":"Azure Files","type":"text","marks":[{"type":"link","attrs":{"href":"https://learn.microsoft.com/en-us/azure/storage/files/storage-files-introduction"}}]},{"text":" (SMB/NFS) as the default persistent storage for data needs to be persisted.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"We use the ","type":"text"},{"text":"Azure Database for PostgreSQL - Flexible Server","type":"text","marks":[{"type":"link","attrs":{"href":"https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/service-overview"}}]},{"text":" service for Usage Engine Private Edition database.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"We assume that you have the ","type":"text"},{"text":"privileges to create user managed identity and grant role","type":"text","marks":[{"type":"strong"}]},{"text":" to the identity created by terraform within the same resource group. See ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/how-manage-user-assigned-managed-identities?pivots=identity-mi-methods-azp"}},{"text":" for more details.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"To use the terraform template, preview feature has to be enabled in your Azure subscription. Refer to ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://learn.microsoft.com/en-us/azure/aks/api-server-vnet-integration#install-the-aks-preview-azure-cli-extension"}},{"text":" for the commands to run. You only need to execute these sections:","type":"text"}]},{"type":"orderedList","attrs":{"order":1},"content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Install the aks-preview Azure CLI extension","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Register the 'EnableAPIServerVnetIntegrationPreview' feature flag","type":"text"}]}]}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Create Basic Cluster and additional infrastructure","type":"text"}]},{"type":"paragraph","content":[{"text":"Follow these steps to create a basic Kubernetes cluster with public and private VPC:","type":"text"}]},{"type":"orderedList","attrs":{"order":1},"content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Extract the downloaded ","type":"text"},{"text":"azure.tar.gz","type":"text","marks":[{"type":"code"}]},{"text":" file.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Go to ","type":"text"},{"text":"/azure/terraform","type":"text","marks":[{"type":"code"}]},{"text":" folder and copy the ","type":"text"},{"text":"terraform.tfvars.example","type":"text","marks":[{"type":"code"}]},{"text":" to ","type":"text"},{"text":"terraform.tfvars","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Edit the ","type":"text"},{"text":"terraform.tfvars","type":"text","marks":[{"type":"code"}]},{"text":" file.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Specify the desired ","type":"text"},{"text":"cluster_name","type":"text","marks":[{"type":"code"}]},{"text":", Azure ","type":"text"},{"text":"location","type":"text","marks":[{"type":"code"}]},{"text":" and ","type":"text"},{"text":"kubernetes_version","type":"text","marks":[{"type":"code"}]},{"text":" (see ","type":"text"},{"text":"Compatibility Matrix (4.2)","type":"text","marks":[{"type":"link","attrs":{"href":"https://infozone.atlassian.net/wiki/spaces/UEPE4D/pages/233279520"}}]},{"text":" to find out which Kubernetes versions that are compatible with this release of Usage Engine Private Edition). Also specify your Azure ","type":"text"},{"text":"resource_group_name","type":"text","marks":[{"type":"code"}]},{"text":", ","type":"text"},{"text":"domain","type":"text","marks":[{"type":"code"}]},{"text":" as well as the desired number of nodes per cluster (","type":"text"},{"text":"user_node_count","type":"text","marks":[{"type":"code"}]},{"text":").","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"If you will be running with a database other than Derby also specify ","type":"text"},{"text":"db_password","type":"text","marks":[{"type":"code"}]},{"text":" and ","type":"text"},{"text":"db_version","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"}]}]}]},{"type":"table","attrs":{"layout":"default","width":760.0,"localId":"3341aa57-056f-49f1-8dc7-9507991ab842"},"content":[{"type":"tableRow","content":[{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1,"colwidth":[238.0]},"content":[{"type":"paragraph","content":[{"text":"terraform.tfvars","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1,"colwidth":[521.0]},"content":[{"type":"paragraph","content":[{"text":"Where to get the value from?","type":"text","marks":[{"type":"strong"}]}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[238.0]},"content":[{"type":"paragraph","content":[{"text":"resource_group_name","type":"text","marks":[{"type":"code"}]}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[521.0]},"content":[{"type":"paragraph","content":[{"text":"Name of the resource group.","type":"text"}]},{"type":"paragraph","content":[{"text":"The resource group must exist in the Azure subscription.","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[238.0]},"content":[{"type":"paragraph","content":[{"text":"location","type":"text","marks":[{"type":"code"}]}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[521.0]},"content":[{"type":"paragraph","content":[{"text":"Location of the cluster","type":"text"},{"type":"hardBreak"},{"text":"Use command ","type":"text"},{"text":"az account list-locations --query \"[].displayName\" -o tsv","type":"text","marks":[{"type":"code"}]},{"text":" to get all available locations.","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[238.0]},"content":[{"type":"paragraph","content":[{"text":"cluster_name","type":"text","marks":[{"type":"code"}]}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[521.0]},"content":[{"type":"paragraph","content":[{"text":"Name of the cluster, it must be unique within the resource group.","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[238.0]},"content":[{"type":"paragraph","content":[{"text":"domain","type":"text","marks":[{"type":"code"}]}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[521.0]},"content":[{"type":"paragraph","content":[{"text":"Your existing domain name. In the Azure management console, this is the ","type":"text"},{"text":"DNS zones ","type":"text","marks":[{"type":"strong"}]},{"text":"that is listed on page ","type":"text"},{"text":"Home | DNS zones | Name","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"},{"type":"hardBreak"},{"text":"The DNS zone must already exist in Azure DNS zones or in other cloud provider DNS zone.","type":"text"}]},{"type":"paragraph","content":[{"text":"The service created by Usage Engine Private Edition in the coming section will be accessible in format ","type":"text"},{"text":"..","type":"text","marks":[{"type":"code"}]},{"text":" i.e. ","type":"text"},{"text":"desktop-online.uepe-aks.example.com","type":"text","marks":[{"type":"strong"}]},{"text":".","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[238.0]},"content":[{"type":"paragraph","content":[{"text":"kubernetes_version","type":"text","marks":[{"type":"code"}]}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[521.0]},"content":[{"type":"paragraph","content":[{"text":"Kubernetes version.","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[238.0]},"content":[{"type":"paragraph","content":[{"text":"user_node_count","type":"text","marks":[{"type":"code"}]}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[521.0]},"content":[{"type":"paragraph","content":[{"text":"Number of nodes in the user node pool.","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[238.0]},"content":[{"type":"paragraph","content":[{"text":"db_password","type":"text","marks":[{"type":"code"}]}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[521.0]},"content":[{"type":"paragraph","content":[{"text":"Choose a secure password for the system database administrator.","type":"text"}]},{"type":"paragraph","content":[{"text":"Minimum 10 characters.","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[238.0]},"content":[{"type":"paragraph","content":[{"text":"db_version","type":"text","marks":[{"type":"code"}]}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[521.0]},"content":[{"type":"paragraph","content":[{"text":"PostgreSQL database version. Supported versions are 11, 12, 13, 14, 15 and 16.","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[238.0]},"content":[{"type":"paragraph","content":[{"text":"db_enabled","type":"text","marks":[{"type":"code"}]}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[521.0]},"content":[{"type":"paragraph","content":[{"text":"Boolean flag to enable PostgreSQL database resource creation.","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[238.0]},"content":[{"type":"paragraph","content":[{"text":"auto_create_ns_record","type":"text","marks":[{"type":"code"}]}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[521.0]},"content":[{"type":"paragraph","content":[{"text":"Boolean flag to enable subdomain NS record auto creation in parent domain. In case your parent domain is not under the same resource group or your parent domain is hosted in another cloud provider, then you ","type":"text"},{"text":"must","type":"text","marks":[{"type":"strong"}]},{"text":" set it to false.","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[238.0]},"content":[{"type":"paragraph","content":[{"text":"storage_enabled","type":"text","marks":[{"type":"code"}]}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[521.0]},"content":[{"type":"paragraph","content":[{"text":"Boolean flag to enable file storage resource creation.","type":"text"}]}]}]}]},{"type":"paragraph","content":[{"text":"Example:","type":"text"}]},{"type":"codeBlock","content":[{"text":"# ____ _____ _____ _____ _ _ _____ ____ _____\n# / ___|| ____|_ _| |_ _| | | | ____/ ___|| ____|_\n# \\___ \\| _| | | | | | |_| | _| \\___ \\| _| (_)\n# ___) | |___ | | | | | _ | |___ ___) | |___ _\n# |____/|_____| |_| |_| |_| |_|_____|____/|_____(_)\n\n# The below values must be set explicitly in order for the setup to work correctly.\n\n# Name of the resource group\n# The resource group must be existed in the Azure subscription.\nresource_group_name = \"PT_Stratus\"\n\n# Location of the cluster\n# Use command `az account list-locations --query \"[].displayName\" -o tsv` to get all available locations.\n# Or see https://azure.microsoft.com/en-us/explore/global-infrastructure/products-by-region/?products=kubernetes-service®ions=all\nlocation = \"Southeast Asia\"\n\n# Name of the cluster, it must be unique within the resource group.\ncluster_name = \"uepe-aks\"\n\n# Domain DNS name\n# The DNS zone must already exist in Azure DNS or in other cloud provider DNS zone.\n# We'll create a subdomain zone from parent domain, the final domain will be in format \".\".\n# Please note that if this domain is hosted on another resource group or other cloud provider, then you must\n# set auto_create_ns_record = false and manually add the subdomain NS record to the parent domain.\ndomain = \"stratus.az.digitalroute.net\"\n\n# Admin user password to the database\ndb_password = \"super_SeCrEt_db_pAsSwOrD_457!\"\n\n# _______ _______ _ _ __ _ ____ _ _____\n# |_ _\\ \\ / / ____| / \\ | |/ / / \\ | __ )| | | ____|_\n# | | \\ \\ /\\ / /| _| / _ \\ | ' / / _ \\ | _ \\| | | _| (_)\n# | | \\ V V / | |___ / ___ \\| . \\ / ___ \\| |_) | |___| |___ _\n# |_| \\_/\\_/ |_____/_/ \\_\\_|\\_\\/_/ \\_\\____/|_____|_____(_)\n\n# The below sections are the default values, tweak them to your needs.\n\n# _ _ ______\n# / \\ | |/ / ___|\n# / _ \\ | ' /\\___ \\\n# / ___ \\| . \\ ___) |\n# /_/ \\_\\_|\\_\\____/\n\n# The size of the Virtual Machine\n# Use command `az vm list-sizes --location \"West Europe\" --output table` to get available VM sizes.\nvm_size = \"Standard_DS2_v2\"\n# Kubernetes version\nkubernetes_version = \"1.29\"\n# Number of nodes in the system node pool, must be at least 1 node.\nsystem_node_count = 2\n# Number of nodes in the user node pool. If zero, the user node pool will not be created.\nuser_node_count = 1\n# DNS prefix specified when creating the managed cluster.\n# It must begin and end with a letter or number, contain only letters, numbers, and hyphens and be between 1 and 54 characters in length.\ndns_prefix = \"aks\"\n# Operating system disk size in GB\nos_disk_size_gb = 30\n# The priority for Virtual Machines within the Virtual Machine Scale Set that powers the node pool.\n# Valid values are Regular and Spot.\npriority = \"Regular\"\n# The eviction policy for Virtual Machines within the Virtual Machine Scale Set that powers the node pool.\neviction_policy = \"Delete\"\n# The maximum price you are willing to pay for Spot VMs. The value must be -1 or a positive value with up to five decimal places.\n# See https://docs.microsoft.com/en-us/azure/virtual-machines/spot-vms for more information.\nspot_max_price = -1\n\n..........\n\n# ____ _ _ ____\n# | _ \\| \\ | / ___|\n# | | | | \\| \\___ \\\n# | |_| | |\\ |___) |\n# |____/|_| \\_|____/\n\n# Specify if nameservers of the subdomain should automatically inserted as a NS record to the parent domain's zone to allow DNS delegation.\n# By default we assume your parent domain zone is hosted on the same resource group, if this is not the case then you MUST set it to false.\nauto_create_ns_record = true\n\n# ____ _____ ___ ____ _ ____ _____\n# / ___|_ _/ _ \\| _ \\ / \\ / ___| ____|\n# \\___ \\ | || | | | |_) | / _ \\| | _| _|\n# ___) || || |_| | _ < / ___ \\ |_| | |___\n# |____/ |_| \\___/|_| \\_\\/_/ \\_\\____|_____|\n\n# Persistent storage\nstorage_enabled = true\n# Kind of account to create. Valid options are Storage, StorageV2, BlobStorage, BlockBlobStorage or FileStorage.\naccount_kind = \"FileStorage\"\n# Tier to use for the storage account. Either Standard or Premium.\n# For BlockBlobStorage and FileStorage accounts only Premium is valid.\naccount_tier = \"Premium\"\n# Storage redundancy type\n# See https://learn.microsoft.com/en-us/azure/storage/files/files-redundancy for available options.\naccount_replication_type = \"LRS\"\n# Protocol used for the share, either SMB or NFS.\n# See https://learn.microsoft.com/en-us/azure/storage/files/storage-files-planning#available-protocols for the comparison.\nenabled_protocol = \"SMB\"\n# Storage capacity in GB\nstorage_capacity = 100\n\n# ____ _ _____ _ ____ _ ____ _____\n# | _ \\ / \\|_ _|/ \\ | __ ) / \\ / ___|| ____|\n# | | | |/ _ \\ | | / _ \\ | _ \\ / _ \\ \\___ \\| _|\n# | |_| / ___ \\| |/ ___ \\| |_) / ___ \\ ___) | |___\n# |____/_/ \\_\\_/_/ \\_\\____/_/ \\_\\____/|_____|\n\n# PostgreSQL database\ndb_enabled = true\n# SKU Name for PostgreSQL, it follows the tier + name pattern.\n# Available tiers are Basic (B), GeneralPurpose (GP), MemoryOptimized (MO).\n# Use command `az postgres flexible-server list-skus --location \"West Europe\" --output table` to find out available SKU names.\n# For example, tier \"GeneralPurpose\" and name \"Standard_D2s_v3\", the SKU name is \"GP_Standard_D2s_v3\".\nsku_name = \"GP_Standard_D2s_v3\"\n# Storage performance tier for IOPS. See https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/postgresql_flexible_server#storage_tier-defaults-based-on-storage_mb for more information\ndb_tier = \"P10\"\n# DB version. Supported versions are 11, 12, 13, 14, 15 and 16.\ndb_version = \"16\"\n# DB storage in MB. See https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/postgresql_flexible_server#storage_tier-defaults-based-on-storage_mb for available storage sizes.\ndb_allocated_storage = 131072\n# Specify which availability zone the instance should be located in.\nzone = 1\n","type":"text"}]},{"type":"panel","attrs":{"panelType":"warning"},"content":[{"type":"paragraph","content":[{"text":"Note!","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"}]},{"type":"paragraph","content":[{"text":"If your parent domain zone is not under the same Resource Group:","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"You must set ","type":"text"},{"text":"auto_create_ns_record = false","type":"text","marks":[{"type":"code"}]},{"text":" to disable subdomain NS record auto creation in the parent domain.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Perform terraform apply.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"When terraform apply is finished, copy the name servers value from terraform output ","type":"text"},{"text":"name_servers","type":"text","marks":[{"type":"code"}]},{"text":" and manually add them to parent domain as NS record. If you are not using Azure DNS as the parent domain, consult your Domain Registrar documentation on how to add NS record.","type":"text"}]}]}]}]},{"type":"orderedList","attrs":{"order":1},"content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Authenticate your computer with Azure by running the following command:","type":"text"}]},{"type":"codeBlock","content":[{"text":"az login --scope https://management.core.windows.net//.default","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Run the following commands:","type":"text"}]},{"type":"codeBlock","content":[{"text":"terraform init\nterraform plan\nterraform apply","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Wait for the terraform commands to finish and you will see an output similar to:","type":"text"}]}]}]},{"type":"codeBlock","content":[{"text":"Apply complete! Resources: 40 added, 0 changed, 0 destroyed.\n\nOutputs:\n\naks_domain_zone_name = \"uepe-aks.stratus.az.digitalroute.net\"\nappgw_ingress_controller_client_id = \"1aee4369-d6c2-4912-ab5a-45fe5d3e3526\"\nappgw_ingress_controller_namespace = \"uepe\"\nappgw_ingress_controller_service_account = \"ingress-azure\"\nappgw_name = \"uepe-aks-application-gateway\"\nazure_resource_group = \"PT_Stratus\"\nazure_subscription_id = \"e6d5f094-a3a1-4ae1-b660-4aa689593bf7\"\nazure_tenant_id = \"bc3dbf22-18e0-4f00-b10d-9595a47abf83\"\ncert_manager_client_id = \"b21cf932-fa5a-4d81-8230-05cfa85245ac\"\ncert_manager_namespace = \"cert-manager\"\ncert_manager_service_account = \"cert-manager\"\ncluster_location = \"Southeast Asia\"\ndb_endpoint = \"uepe-aks-db.postgres.database.azure.com\"\ndb_password = \"super_SeCrEt_db_pAsSwOrD_457!\"\ndb_username = \"postgres\"\nexternal_dns_client_id = \"c97fad12-109e-4fb4-971e-9446b0964f3c\"\nexternal_dns_namespace = \"uepe\"\nexternal_dns_service_account = \"external-dns\"\nkube_config = \"./kubeconfig\"\nkubernetes_cluster_name = \"uepe-aks\"\nname_servers = toset([\n \"ns1-38.azure-dns.com.\",\n \"ns2-38.azure-dns.net.\",\n \"ns3-38.azure-dns.org.\",\n \"ns4-38.azure-dns.info.\",\n])\nstorage_account_key = \"trfSDOsyDS2SFCp59Q3x4t3RA8a2KZGtNUJ2lLJ7xS0L6pDm8BZ2AFkoCoMHmtUizQSSLOA+8nQA+AStvjbRaw==\"\nstorage_account_name = \"uepeaks\"\nstorage_capacity_gb = 100\nstorage_persistence_yaml = \"./manifests/storage_persistence.yaml\"\nstorage_share_name = \"uepe-aks-storage-share\"","type":"text"}]},{"type":"panel","attrs":{"panelType":"note"},"content":[{"type":"paragraph","content":[{"text":"Ensure that you save the output from terraform above since it will be used as input throughout the remainder of this installation guide.","type":"text"}]}]},{"type":"paragraph","content":[{"text":"A basic Kubernetes cluster has now been set up successfully.","type":"text"},{"type":"inlineExtension","attrs":{"extensionType":"com.atlassian.confluence.macro.core","extensionKey":"excerpt-include","parameters":{"macroParams":{"":{"value":"General Kubernetes Preparations"},"name":{"value":"terraform state message"},"nopanel":{"value":"true"}},"macroMetadata":{"macroId":{"value":"fb3619c0-42cf-4162-aba9-298b8a8a1c64"},"schemaVersion":{"value":"1"},"title":"Insert excerpt"}},"localId":"04dd1edd-974a-455a-8f3d-714a48c77591"}}]},{"type":"paragraph","content":[{"text":"In this example, an PostgreSQL flexible servers database instance up and running on private subnet VNET with default listening port 5432. The default database ","type":"text"},{"text":"postgres","type":"text","marks":[{"type":"code"}]},{"text":" is accessible within the cluster at end point ","type":"text"},{"text":"uepe-aks-db.postgres.database.azure.com","type":"text","marks":[{"type":"code"}]},{"text":" with admin username ","type":"text"},{"text":"postgres","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"}]},{"type":"paragraph","content":[{"text":"You can check the status of the cluster, db and the other resources in the Azure dashboard.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Configure Cluster Access","type":"text"}]},{"type":"paragraph","content":[{"text":"The previous ","type":"text"},{"text":"terraform","type":"text","marks":[{"type":"code"}]},{"text":" command will generate a ","type":"text"},{"text":"./kubeconfig","type":"text","marks":[{"type":"code"}]},{"text":" file containing information on how to connect to your newly created cluster. Make sure to set the ","type":"text"},{"text":"KUBECONFIG","type":"text","marks":[{"type":"code"}]},{"text":" environment variable to point to that file on the terraform’s working directory:","type":"text"}]},{"type":"codeBlock","content":[{"text":"export KUBECONFIG=","type":"text"}]},{"type":"paragraph","content":[{"text":"This will ensure that tools like ","type":"text"},{"text":"kubectl","type":"text","marks":[{"type":"code"}]},{"text":" and ","type":"text"},{"text":"helm","type":"text","marks":[{"type":"code"}]},{"text":" will connect to your newly created cluster.","type":"text"}]},{"type":"paragraph","content":[{"text":"Alternatively, you can use the following command to bind your local kubectl to the created cluster:","type":"text"}]},{"type":"codeBlock","content":[{"text":"az aks get-credentials --name uepe-aks --resource-group PT_Stratus","type":"text"}]},{"type":"paragraph","content":[{"text":"You can check the status of the cluster nodes with the following command:","type":"text"}]},{"type":"codeBlock","content":[{"text":"kubectl get nodes","type":"text"}]},{"type":"paragraph","content":[{"text":"For this example cluster the output will look something like this:","type":"text"}]},{"type":"codeBlock","content":[{"text":"NAME STATUS ROLES AGE VERSION\naks-default-41718565-vmss000000 Ready 6d2h v1.29.5\naks-default-41718565-vmss000001 Ready 6d1h v1.29.5\naks-internal-23883603-vmss000000 Ready 6d2h v1.29.5","type":"text"}]},{"type":"paragraph"},{"type":"paragraph","content":[{"type":"inlineExtension","attrs":{"extensionType":"com.atlassian.confluence.macro.core","extensionKey":"excerpt-include","parameters":{"macroParams":{"":{"value":"General Kubernetes Preparations"},"name":{"value":"common-namespace"},"nopanel":{"value":"true"}},"macroMetadata":{"macroId":{"value":"89c735f7-e052-4e79-bf3f-78bb81c8baff"},"schemaVersion":{"value":"1"},"title":"Insert excerpt"}},"localId":"2b8d2e13-f23e-454b-8d51-691b90cf213a"}}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Persistent Volume and Persistent Volume Claim","type":"text"}]},{"type":"panel","attrs":{"panelType":"warning"},"content":[{"type":"paragraph","content":[{"text":"Note!","type":"text","marks":[{"type":"strong"}]}]},{"type":"paragraph","content":[{"text":"The persistent volume setup is an optional step. Skip this section if you do not intend to have persistent file storage.","type":"text"}]}]},{"type":"paragraph","content":[{"text":"Persistent Volume (PV) and Persistent Volume Claim (PVC) must be setup before Usage Engine Private Edition Helm Chart installation. The PV and PVC yaml files have already been generated in ","type":"text"},{"text":"/manifests/storage_persistence.yaml","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"}]},{"type":"paragraph","content":[{"text":"Kubernetes needs credentials to access the Azure storage that was created by the ","type":"text"},{"text":"terraform","type":"text","marks":[{"type":"code"}]},{"text":" command in the previous section. Create the storage account secret with this command:","type":"text"}]},{"type":"codeBlock","content":[{"text":"kubectl create secret generic -persistence-secret \\\n--from-literal=azurestorageaccountname= \\\n--from-literal=azurestorageaccountkey= \\\n-n uepe","type":"text"}]},{"type":"paragraph","content":[{"text":"Change directory to ","type":"text"},{"text":"/manifests","type":"text","marks":[{"type":"code"}]},{"text":".","type":"text"}]},{"type":"paragraph","content":[{"text":"To setup Persistent Volume and Persistent Volume Claim run the following command:","type":"text"}]},{"type":"codeBlock","content":[{"text":"kubectl apply -f storage_persistence.yaml -n uepe","type":"text"}]},{"type":"paragraph","content":[{"text":"To ensure PVC bounded to allocated Persistent Volume run the following command:","type":"text"}]},{"type":"codeBlock","content":[{"text":"kubectl get pvc -n uepe","type":"text"}]},{"type":"paragraph","content":[{"text":"The output should display PVC status is Bound, similar to:","type":"text"}]},{"type":"codeBlock","content":[{"text":"NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE\nuepe-aks-pvc Bound uepe-aks-pv 100Gi RWX azurefile-csi 6s","type":"text"}]},{"type":"paragraph","content":[{"text":" ","type":"text"}]},{"type":"paragraph","content":[{"text":"This section is now complete and you can proceed to the ","type":"text"},{"text":"Kubernetes Cluster Add-ons - Azure (4.2)","type":"text","marks":[{"type":"link","attrs":{"href":"https://infozone.atlassian.net/wiki/spaces/UEPE4D/pages/249233473"}}]},{"text":" section.","type":"text"}]}],"version":1}

Browser not supported