Flushing Metered Records

Cloud_Edition_button.webp

The process of sending data from the Meter processor to downstream systems is known as ‘flushing a record’. The flushed record is in JSON format and it will be sent for each meter associated with the Meter processor.

Regardless of whether the meter was updated or not during the last metering period, the meter will always be flushed when its period ends. Meters will only be flushed after the Active to date time has passed if they have been updated during the current metering period.

A usage audit is generated when the input data is sent out from the Meter processor.

The metered record can be viewed in JSON format in the

view-session.png
View meter dialog

The following fields are included in the record sent to downstream systems in JSON format:

Field

Purpose

Field

Purpose

id

The unique ID of the meter.

userId

The unique ID of the user the meter belongs to, for example an email address, user name, or MSISDN number.

meterTypeId

The unique ID of the meter type that the meter aggregates usage for.

meterTypeName

The name of the meter type this meter is using.

timezone

The timezone that should be applied for reset time. Defined in the meter type for this meter.

meterKey

The meter key for the Meter processor aggregating the usage for the meter.

value

The current value of the meter, updated based on the guiding rules and aggregation type defined in the meter type.

unit

The unit in which this meter measures usage. Defined in the meter type for this meter.

createdAt

The UTC timestamp for when the meter was created.

updatedAt

The UTC timestamp for when the meter was last updated.

periodStart

The UTC timestamp for when the metering period started. This is calculated based on when the last period ended.

periodEnd

The UTC timestamp for when the metering period ended. This is calculated based on the reset schedule.

groups

A granular breakdown of the aggregated values, grouped by the guiding rules defined in the . Each group is represented by an array containing the parameters key, fields, and value.

carryFirst

The first occurrence of one or more fields that should be carried over from the usage record. This is configured in Other operations in the .

carryLast

The last occurrence of one or more fields that should be carried over from the usage record. This is configured in Other operations in the .

deleteOnReset

If this is set to true, the meter will be deleted when the ongoing metering period ends. This can be updated using the update meter API.

meterMetaData

This is additional meta information related to the usage ingestion example, datetime when the first and last record matched this meter.

Example - Metered Output Record

{ "carryFirst": { "country": "Norway" }, "carryLast": { "timestamp": "2023-07-05T22:01:04.431Z" }, "createdAt": "2023-07-01T00:00:00.000Z", "deleteOnReset": false, "groups": [ { "fields": { "API name": "createUser" }, "key": "API name:createUser", "value": 10 }, { "fields": { "API name": "updateCounter" }, "key": "API name:updateCounter", "value": 15 } ], "id": "8a76066d-ff29-417c-a18e-abde22e44310", "meterKey": "a1f8797b-c827-4483-b6cf-6d3d3baa18a7", "meterMetaData": { "firstEvent": "2023-07-01T13:37:11.111Z", "lastEvent": "2023-07-05T22:01:04.431Z" }, "meterTypeId": "645ed240-02b5-400c-9a3c-21857e8f2ac4", "meterTypeName": "Usage based API counter", "periodEnd": "2023-08-01T00:00:00.000Z", "periodStart": "2023-07-01T00:00:00.000Z", "timezone": "Etc/UTC", "unit": "requests", "updatedAt": "2023-07-05T22:01:04.431Z", "userId": "user0@example.com", "value": 25 }

Meter Reset

When a flush happens, the values of the meters are also reset to their initial value for example, a value is set to 0, or groups are removed to empty the bucket. At the same time, meters are prepared for the next metering period and Period start and Period end are updated based on when the flush happens and the Reset schedule.

If the meters are flushed by Period end in the , then the new Period start is set to when the last period ended, and the Period end is calculated based on the Reset schedule of the respective meter type.

Example - Meter Type has Reset Schedule every day at Midnight

Ongoing period:

Period start - Jan 1 00:00 Period end - Jan 2 00:00

Scenario 1: A stream runs on Jan 2 at 02:00. The new period is: Period start - Jan 2 00:00 Period end - Jan 3 00:00

Scenario 2: A stream does not run on Jan 2 at 00:00, but runs on Jan 3 at 02:00. The new period is: Period start - Jan 2 00:00 and Period end - Jan 4 00:00.

This happens because before the incoming usage data is processed, the stream scans for meters whose periods have ended and flushes them. In scenario 2 where the stream runs on Jan 3, the usage data from Jan 2 after midnight is included in the new period, and usage data up until Jan 2 at midnight is already aggregated in the metered and flushed record.

If the meters are flushed by Stream execution in the , the Period end of the flushed metered record is set to the time of the flush. The new Period start of the meter in the stream is also set to the time of the flush, and the Period end is calculated using the meter type Reset schedule.

Period start and Period end for a meter can be checked in the .

Â