Date configuration
The Date function is a processor that adds, reformats, or modifies a date field. The configuration is made up of the Date field section, where you can select one of three key actions to take on the selected Date field:
Modify
Copy and modify
Add new
Depending on which action you select, the available configuration options will change slightly.
Note!
Make sure the date format is consistent across your data before using the Date function. If the function encounters an incorrect format or a null value, the stream will fail.
Tokens used to format or parse date and time strings
Use the tokens below to define how date and time format should be written in your configuration.
Input | Example | Description |
|---|---|---|
| 01 | Two-digit year |
| 2001 | Four-digit year |
| 1-12 | Month, beginning at 1 |
| 01-12 | Month, 2-digits |
| Jan-Dec | The abbreviated month name |
| January-December | The full month name |
| 1-31 | Day of month |
| 01-31 | Day of month, 2-digits |
| 0-23 | Hours |
| 00-23 | Hours, 2-digits |
| 1-12 | Hours, 12-hour clock |
| 01-12 | Hours, 12-hour clock, 2-digits |
| 0-59 | Minutes |
| 00-59 | Minutes, 2-digits |
| 0-59 | Seconds |
| 00-59 | Seconds, 2-digits |
| 0-9 | Hundreds of milliseconds, 1-digit |
| 00-99 | Tens of milliseconds, 2-digits |
| 000-999 | Milliseconds, 3-digits |
| -05:00 | Offset from UTC |
| -0500 | Compact offset from UTC, 2-digits |
| AM PM | Post or ante meridiem, upper-case |
| am pm | Post or ante meridiem, lower-case |
| 1st... 31st | Day of Month with ordinal |
| 1410715640.579 | Unix timestamp |
| 1410715640579 | Unix ms timestamp |
Note!
When selecting an output format, you can either choose from the dropdown list of common formats or type in a custom format. If you want the timezone marker 'Z' (which represents UTC) to appear in the output, make sure to select one of the options at the bottom of the list that includes [Z].
Modify
Use this when you need to reformat a date/time field and/or apply a date operation.
Example - Modify the format and add a date operation to a date/time field
Say your input date is formatted as YYYY-MMM-MM(e.g., 2022-Jan-01), and you want to:
Reformat it to ISO 8601 with milliseconds and a time zone indicator,
YYYY-MM-DDTHH:mm:ss.SSS[Z]Modify the time units (hour, minute, second) so that they reflect 23:59:59 (while keeping the date part unchanged)
To do this, you would:
Change the field’s output format to
YYYY-MM-DDTHH:mm:ss.SSS[Z].Add three time operations (one for hour, minute and second) to add the updated time to the modified timestamp.
An example of a final output would be: 2022-01-03T23:59:59.000Z
Setting | Description |
|---|---|
Field | Select the field you want to modify from the dropdown list, or create a field using the Add field option.  ‘Field’ dropdown list |
Input format | Add the input format of the field you are copying/modifying. You can select the input format from the dropdown list, or add it using the + Add format button at the bottom of the list. |
Output format | Select the output format for the new or modified field this is the format you want the field to be converted into. You can choose a format from the dropdown list or add a custom format using the + Add format button at the bottom. |
Use input format | You can check this box to use the Input format in the Output field. This is useful if you are not modifying the field, but only adding a operation to it. |
Copy and modify
Use this when you want to keep the original date/time field and create a new, modified date/time field.
Example - Copy and modify an existing date field
Lets say your current field timestamp is formatted as YYYY-MMM-DD (e.g. 2022-Jan-01), and you want to create a new field called timeout in the format YYYY-MM-DDTHH:mm:ss.SSSZ (e.g. 2022-01-01).
To do this, you would:
Update the Source field to 'timestamp'
Update the Target field to ‘timeout’
Change the field’s output format to
YYYY-MM-DDTHH:mm:ss.SSSZ
A new field is created with the final output: 2022-01-01T00:00:00.000+00:00
Configuration option | Description |
|---|---|
Source field | Select a source field from the drop down list. This is the field that you want to copy and modify, for example you want to use the current timestamp field to make a copy and then modify the copy. You will end up with two timestamp fields. |
Target field | Create a Target field, this will be the new field that is created with the modified format. |
Input format | Add the input format of the field you are copying/modifying. You can select the input format from the dropdown list, or add it using the Add format button at the bottom of the list. |
Output format | Select the output format for the new field this is the format you want the field to be converted into. You can choose a format from the dropdown list or add a custom format using the Add format button at the bottom. |
Use input format | You can check this box to use the Input format in the Output field. This is useful if you are not modifying the field, but only adding a operation to it. |
Add new
Use this when you want to create a new date/time field from scratch.
Note!
The newly added date will default to the current UTC time if no date operation is applied.
Example - Add a new date field
Lets say you want to create a date field for your data from scratch with the format YYYY-MM-DD and you want it to reflect the 1st day of the current month and year (e.g. 2025-04-01)
To do this, you would:
Update the New field name to 'date'
Update the Output format to
YYYY-MM-DDAdd a Date operation for ‘Current year’
Add a Date operation for ‘Current month'’
Add a Date operation to set a value on Day to be the 1st.
A new field is created with the final output: 2025-04-01 for all records
Setting | Description |
|---|---|
New field name | Add the name of the new field that you want to create. |
Output format | Select the output format for the new or modified field this is the format you want the field to be converted into. You can choose a format from the dropdown list or add a custom format using the + Add format button at the bottom. |
Date operation
You can add date/time operations to any existing field whether you're already modifying or copying the field, or you're leaving it unchanged.
For example, you might want to make a field reflect the last day of the current month across all records, even if you're not making any other changes to its format.
To do this, click + Add date operation, then select the Time unit and Operation to apply.
Note!
If you select Add a value, Subtract a value, or Set a value, a field will appear where you can enter the value.
Example - Adding a Date operation
You want the timestamp field to always reflect the last day of the month.
In the Time unit dropdown: select Day
In the Operation dropdown: select Last day of the month.
+ Add field
Use this button to add, reformat, or modify another date field, for example, a second date/time field that you want to modify independently from the first. There is a max character limit of 180 for this field.
Note!
You will see a Duplicate item warning if you try to modify the same field more than once.