REMOVE THIS SECTION AS IT'S AN EXACT REPETITION OF THE PREVIOUS, WHICH MAKES IT CONFUSING To create a new Parquet profile configuration, click the New Configuration button in the upper left part of the Desktop window, and then select Parquet Profile from the menu. The contents of the menus in the menu bar may change depending on which configuration type has been opened. The Parquet profile uses the standard menu items and buttons that are visible for all configurations, and these are described in 2.1 Menus and Buttons.
PLEASE PROVIDE MORE INFORMATION IN THIS SECTION - WHAT DO WE WANT TO DESCRIBE HERE
The Parquet profile's Schema tab with an example of a defined Schema. You will have to write the Schema for your desired functions.
| Setting | Description |
|---|---|
Schema | See below. |
Validate | Press the Validate button to validate the Schema and make sure it has a correct format. |
Defining the Parquet Schema
To be able to define a Schema, it is useful to have knowledge about primitives, nested groups, repetition levels, and logical types, as described below:
Apache Parquet supports a small set of primitives (integer, floating point, boolean, and byte array). These primitives can be extended using logical type annotations which are modifiers on primitives. For example, the UTF8 annotation is a modifier to byte arrays that denotes string data. Parquet also supports structured data through groups and repetitions (that is, optional, required, repeated). Example - Parquet Schema This structured text block shows an example Parquet schema for company employees:Example Parquet Schema
message employee {
required group id {
required group name {
required binary surname (UTF8);
required binary firstName (UTF8);
optional binary preferredFirstName (UTF8);
}
required int32 employeeNumber;
}
optional group phones (LIST) {
repeated group list {
required group element {
required binary type (ENUM);
required binary phoneNumber (UTF8);
}
}
}
required binary email (UTF8);
optional binary manager (UTF8);
required binary jobTitle (UTF8);
required group team {
required binary country (UTF8);
required binary businessUnit (UTF8);
required binary function (UTF8);
optional binary team (UTF8);
optional binary department (UTF8);
required binary legalEntity (UTF8);
}
optional int32 birthdate (DATE);
}