Creating Custom Metrics on Prometheus Adapter(3.1)

Custom Metrics, Scaling, Prometheus Adapter Examples

The Prometheus adapter is what makes Prometheus metrics available as metrics in Kubernetes. Using the adapter together with the metrics exposed by  it is possible to, for example, scale deployments based on these custom metrics.

Creating New Metrics using the Adapter

Sometimes you need to take a metrics value and create a new metric based on the original one. For example if you want to do a sliding window or defining a new metric that shows the rate or rates from another metric.

The default  Adapter metric rule definitions used during installation are the following:

- seriesQuery: '{__name__=~"^com_digitalroute.*"}' 
  resources: 
    overrides:
      namespace: {resource: "namespace"}
      pod: {resource: "pod"}
  name: 
    matches: ^(.*) 
    as: ""

  metricsQuery: sum(<<.Series>>{<<.LabelMatchers>>}) by (<<.GroupBy>>)

This definition makes all metrics starting with com_digitalroute available as “pod” metrics in Kubernetes.

Below is an example for creating a new custom metric. We will be using the Workflow Throughput MIM value found in  to generate our custom metrics to publish to kubernetes.

- seriesQuery: '{__name__=~"com_digitalroute_wf_Workflow_Workflow_Throughput"}' 
  resources:
    overrides:
      namespace: {resource: "namespace"}
      pod: {resource: "pod"}
  name: 
    matches: ^(.*) 
    as: "workflows_throughput_avg"
  metricsQuery: sum(avg_over_time(com_digitalroute_wf_Workflow_Workflow_Throughput[5m])) by (<<.GroupBy>>) 

You can read more about creating metrics via the adapter configuration from https://github.com/kubernetes-sigs/prometheus-adapter.