Overview
Use the Validate processor-type Function to validate the data from any collector or processor.
The process works by adding a JSON schema to determine if a record is valid or not. The JSON schema describes the structure or content, or both, of the record and discards records that do not match the schema. Validation fields are searchable.
There are two ways of interacting with this node – a Rules selection using a dropdown menu entry or by placing code in the Schema section.
Rules Selection
There are two subsections available in the JSON Rules selection: Fields and Objects.
There is a distinction between the JSON objects and fields according to the format specifications.
Objects are data encapsulated in the JSON string format that contain key/value pairs of information. The keys themselves must be of the “string” data type and the values must correspond to a valid JSON data type. Fields denote the name/value pairs that are placed in the objects.
Using the relevant “Add Property” or “Add Object” the desired items can be added to the service.
Field Rules Selection
When the “New Field” button is clicked the users can select the required Field from the dropdown menu selection. A description can be assigned to each property as well.
The user interface allows customers to choose between adding in the Output of the previous function or manually specify a field property of their choice by toggling the desired option.
Validation rules can be entered by clicking on the “Add Validation Rule” button. This adds a row, comprising a Key dropdown menu choice with an assigned Value input field box. Additional lines can be creating created by pressing the “Add Validation Rule” button again, while existing rules can be removed by clicking on the remove icon on the selected row.
Data Correction
The Send invalid data correction button is used to enable or disable data correction. Refer to Data Correction for more information.
Validation Types
The Validate node can process instances of different JSON schema types. The current JSON schema implementation supports all data types (number, integer, string, boolean, array, object, or null) and in accordance to the official specifications.
The Validation Function is interactive, if incorrect values are entered, an appropriate error will be shown.
The Validate Function uses Normalization Rules during validation. The following validation rules apply:
- The Validate Function does not perform strict validation on the input data types
- The record that passes through the validation function might be converted to the data type that is defined in the JSON schema.
- Validation is done when the input data type can be converted