Autoscaling your ECD with Workflow Throughput(3.1)

Owned by Jenni Wallin
May 10, 2023
3 min read

The steps below will provide an example for creating an EC Deployment that automatically scales using the workflow throughput metrics. Follow these steps to successfully create your EC Deployment. It will take some time for the container and the EC to be set up after you have successfully created the EC Deployment.

Prerequisites

You will need ensure that the Prometheus server and the Prometheus Adapter is installed before proceeding with the steps. If it is not installed yet, get an Admin user to run through the Prometheus installation guide in Setting up Prometheus(3.0).

You will also need to set up the custom metrics for the Prometheus Adapter to publish the workflow throughput metrics for Kubernetes and the EC Deployment to retrieve the metrics. Get an Admin user to configure the rules following the examples show in Creating Custom Metrics on Prometheus Adapter(3.0).

Create a simple dynamic workflow that will collect, process and output a file. You will want the workflow to be able to increase in throughput as it works so that it triggers the autoscaling.

Creating the EC Deployment

  1. Click on Create an EC Deployment.
  2. Enter an EC Deployment name.
  3. You can leave the Memory and CPU Configuration as its default as well as leave the Advanced Settings with the default value.
  4. Click on the Next button to go to the Workflow section.
  5. Add the simple dynamic workflow that you have created in your prerequisite step. For now add only one workflow.
  6. Click on the Next button to go to the Network section.
  7. You can skip the Network section for this example.
  8. Review the summary of your ECD. Ensure that you have added the workflow.
  9. Click on Create and allow the ECD start running.
  10. You can click on your EC Deployment card in the EC Deployment Overview page to check if your workflow is running.

Configuring your EC Deployment to auto scale

Because the metrics we are using is tied to the workflow throughput, we will first need the workflow to run before the metrics data is generated. So once your workflow is started up and running, we can now proceed with configuring your EC Deployment with auto scaling.

  1. On your EC Deployment card in the EC Deployment Overview page, click on the Edit button.
  2. Enable Auto Scaling by clicking on the Enable checkbox.
  3. Configure your Min and Max Replica. For this example, we left it as the default value.
  4. Under Scaling Metric(s) section, Click on Add/Edit Metrics.
  5. Search for and select the Workflow Throughput metrics, it should look something like com_digitalroute_wf_Workflow_Workflow_Throughput. Add the average throughput value that you want to set as the threshold for the EC Deployment to begin auto scaling. If you have properly configured the Prometheus Adapter with the Workflow Throughput custom metrics, you should be able to search for the value. However do give it some time for the workflow to execute and for the Prometheus server to scrape for the metrics.
  6. You can remove the default CPU Utilization metrics after adding your own custom metrics.
  7. Continue through the Workflow and Network sections.
  8. Review the summary of your changes. The Min and Max replica values should be at set to 1 and 4, which is the default. Check to see that your custom metrics and the average value is correct.
  9. Click on Update to confirm the changes to your EC Deployment.
  10. Restart your EC Deployment for the changes to take effect.

You should be able to see the EC Deployment scale as the average value threshold is reached by your workflow's throughput. If the throughput value does not increase, add more files or workflows to it to increase the value.