Flushing Metered Records
The process of sending data from the Meter processor to downstream systems is known as ‘flushing a record’. The flushed record is in JSON format and it will be sent for each meter associated with the Meter processor.
Regardless of whether the meter was updated or not during the last metering period, the meter will always be flushed when its period ends. Meters will only be flushed after the Active to date time has passed if they have been updated during the current metering period.
A usage audit is generated when the input data is sent out from the Meter processor.
The metered record can be viewed in JSON format in the Meter Inspector
The following fields are included in the record sent to downstream systems in JSON format:
Field | Purpose |
---|---|
id | The unique ID of the meter. |
userId | The unique ID of the user the meter belongs to, for example an email address, user name, or MSISDN number. |
meterTypeId | The unique ID of the meter type that the meter aggregates usage for. |
meterTypeName | The name of the meter type this meter is using. |
timezone | The timezone that should be applied for reset time. Defined in the meter type for this meter. |
meterKey | The meter key for the Meter processor aggregating the usage for the meter. |
value | The current value of the meter, updated based on the guiding rules and aggregation type defined in the meter type. |
unit | The unit in which this meter measures usage. Defined in the meter type for this meter. |
createdAt | The UTC timestamp for when the meter was created. |
updatedAt | The UTC timestamp for when the meter was last updated. |
periodStart | The UTC timestamp for when the metering period started. This is calculated based on when the last period ended. |
periodEnd | The UTC timestamp for when the metering period ended. This is calculated based on the reset schedule. |
groups | A granular breakdown of the aggregated values, grouped by the guiding rules defined in the Meter Configuration. Each group is represented by an array containing the parameters |
carryFirst | The first occurrence of one or more fields that should be carried over from the usage record. This is configured in Other operations in the Meter Configuration. |
carryLast | The last occurrence of one or more fields that should be carried over from the usage record. This is configured in Other operations in the Meter Configuration. |
deleteOnReset | If this is set to true, the meter will be deleted when the ongoing metering period ends. This can be updated using the update meter API. |
meterMetaData | This is additional meta information related to the usage ingestion example, datetime when the first and last record matched this meter. |
Example - Metered Output Record
{
"carryFirst": {
"country": "Norway"
},
"carryLast": {
"timestamp": "2023-07-05T22:01:04.431Z"
},
"createdAt": "2023-07-01T00:00:00.000Z",
"deleteOnReset": false,
"groups": [
{
"fields": {
"API name": "createUser"
},
"key": "API name:createUser",
"value": 10
},
{
"fields": {
"API name": "updateCounter"
},
"key": "API name:updateCounter",
"value": 15
}
],
"id": "8a76066d-ff29-417c-a18e-abde22e44310",
"meterKey": "a1f8797b-c827-4483-b6cf-6d3d3baa18a7",
"meterMetaData": {
"firstEvent": "2023-07-01T13:37:11.111Z",
"lastEvent": "2023-07-05T22:01:04.431Z"
},
"meterTypeId": "645ed240-02b5-400c-9a3c-21857e8f2ac4",
"meterTypeName": "Usage based API counter",
"periodEnd": "2023-08-01T00:00:00.000Z",
"periodStart": "2023-07-01T00:00:00.000Z",
"timezone": "Etc/UTC",
"unit": "requests",
"updatedAt": "2023-07-05T22:01:04.431Z",
"userId": "user0@example.com",
"value": 25
}
Meter Reset
When a flush happens, the values of the meters are also reset to their initial value for example, a value is set to 0, or groups are removed to empty the bucket. At the same time, meters are prepared for the next metering period and Period start and Period end are updated based on when the flush happens and the Reset schedule.
If the meters are flushed by Period end in the Meter Processor , then the new Period start is set to when the last period ended, and the Period end is calculated based on the Reset schedule of the respective meter type.
Example - Meter Type has Reset Schedule every day at Midnight
Ongoing period:
Period start - Jan 1 00:00 Period end - Jan 2 00:00
Scenario 1: A stream runs on Jan 2 at 02:00. The new period is: Period start - Jan 2 00:00 Period end - Jan 3 00:00
Scenario 2: A stream does not run on Jan 2 at 00:00, but runs on Jan 3 at 02:00. The new period is: Period start - Jan 2 00:00 and Period end - Jan 4 00:00.
This happens because before the incoming usage data is processed, the stream scans for meters whose periods have ended and flushes them. In scenario 2 where the stream runs on Jan 3, the usage data from Jan 2 after midnight is included in the new period, and usage data up until Jan 2 at midnight is already aggregated in the metered and flushed record.
If the meters are flushed by Stream execution in the Meter Processor , the Period end of the flushed metered record is set to the time of the flush. The new Period start of the meter in the stream is also set to the time of the flush, and the Period end is calculated using the meter type Reset schedule.
Period start and Period end for a meter can be checked in the Meter Inspector.
Â