Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

The ParquetDecoderUDR includes a schema field that describes the schema of the payload. This schema makes use of ParquetType to describe fields.


Field
Description
typename

Field data type – APL type that corresponds naturally to Parquet schema type.

Enum (APL)

Corresponding Parquet

string

binary (UTF8)
binary (JSON)
binary (ENUM)

bytearray

binary
fixed_len_byte_array
int96

boolean

boolean

double

double

float

float

int

int32
int32 (TIME_MILLIS)

long

int64
int64 (TIME_MICROS)

date

int32 (DATE)
int64 (TIMESTAMP_MILLIS)
int64 (TIMESTAMP_MICROS)

map

nested group with “required” or “optional” repetition

list

“repeated” repetition for any primitive or nested group
       OR
nested LIST type specified in Logical Types

generalmap

nested MAP type specified in Logical Types

requiredBoolean that indicates whether this field is guaranteed to be present.
originaltypenameString that identifies the Logical Type definition (if any) from the Parquet schema.
mapschemaMap from string to ParquetType. When the typename field is “map”, this field will describe the nested structure.
listschemaList of ParquetType. When the typename field is “list”, this field will describe the type of the list.
  • No labels