Versions Compared

Old Version 63

changes.mady.by.user Former user

Saved on

compared with

New Version 64

changes.mady.by.user Former user

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

 

The Data Aggregator Function consolidates related records that originate from either a single or multiple sources, into a single record, and and turns it into a more useful format for analysis and decision-making. Related records are grouped into sessions according to the value of their respective fields, and a set of configurable conditions.

...

FieldDescription
Fields

Specify the fields (keys) to be used for grouping aggregated data. If a record has the same values as another record in the selected fields, they will be grouped in the same session.

Info
titleExample

For example, there are three records being read in a stream with field key, user and you decide to group by user

Record1 has user field with value set as 1

Record2 has user field with value set as 2

Record3 has user field with value set as 1

So, Record1 and Record3 will be grouped in the same aggregated data session and the value will be updated to 2 (in case of the SUM operation). And, Record 2 will be in a separate session.


You can either type the name of the field(s) or select the field(s) from the drop-down menu that appears when you click on the field. In the Fields drop-down menu, you also have the option to Select all or Deselect all fields.

Group based on date/time

Select the checkbox to enable aggregation based on a time period. Specify applicable fields, containing timestamp data, to be used for grouping of aggregation data.

Select the Period to group the information by:

  • Hour
  • Day
  • Month
  • Year
Info
titleTime Period Format

The supported time period format is ISO 8601 (extended format) in accordance with the YYYY-MM-DDTHH:mm:ss.sssZ pattern.

In case you need to convert a date format to support the ISO standard, see Script.

An example would be: "2019-01-13T00:00:00.000Z". 

Click  to add additional grouping criteria based on the time period.


Note
titleConfiguration

Any change in the Group Fields configuration may result in a new aggregation session, however, the old sessions will still be in the storage. So if you change it back to the previous configuration, the already existing session in the storage (if any) will be used during the aggregation. Refer to the note for more information on the TTL of an aggregation session.

Aggregate Fields
Anchor
aggregate fields
aggregate fields

...

Property name Description

count

Number of aggregated records

flushType

The reason for session being flushed out. Shows any of the values: ALL_FILES, EACH_FILE, TIMEOUT and CONDITION. During preview, the value will be empty.

firstEventDate and time of the first aggregated record in the session
lastEvent

Date and time of the last aggregated record in the session


Infonote
titleNote!

Licensing Information

Some functionality functionality described in this documentation are optional and might not be part of the edition covered by your company's license.

...