ParquetType

The ParquetDecoderUDR includes a schema field that describes the schema of the payload. This schema makes use of ParquetType to describe fields.

Field

Description

typename

Field data type – APL type that corresponds naturally to Parquet schema type.

Enum (APL)	Corresponding Parquet
string	binary (UTF8) binary (JSON) binary (ENUM)
bytearray	binary fixed_len_byte_array int96
boolean	boolean
double	double
float	float
int	int32 int32 (TIME_MILLIS)
long	int64 int64 (TIME_MICROS)
date	int32 (DATE) int64 (TIMESTAMP_MILLIS) int64 (TIMESTAMP_MICROS)
map	nested group with “required” or “optional” repetition
list	“repeated” repetition for any primitive or nested group OR nested LIST type specified in Logical Types
generalmap	nested MAP type specified in Logical Types

required Boolean that indicates whether this field is guaranteed to be present.

originaltypename String that identifies the Logical Type definition (if any) from the Parquet schema.

mapschema Map from string to ParquetType. When the typename field is “map”, this field will describe the nested structure.

listschema List of ParquetType. When the typename field is “list”, this field will describe the type of the list.

0 Comments