...
| Setting | Description | ||
|---|---|---|---|
Compression Codec | Uncompressed Snappy | Block Size | The compression algorithm used to compress pages when encoding. Valid choices are Uncompressed, Snappy, Gzip, Lzo, Brotli, Lz4, and Zstd. |
Block Size | The Block Size is the size of a row group buffered in memory. Block size limits the memory usage when writing. Larger values will improve I/O when reading but consume more memory when writing. Default Block Size is set to 134217728. | ||
| Page Size | The page is the smallest unit that must be read fully to access a single record. When reading, each page can be decompressed independently. If this value is too small, the compression will deteriorate. Default is 1048576. | ||
| Dictionary Page Size | There is one dictionary page per column per row group when dictionary encoding is used. Dictionary page size works like the page size but for dictionary encoding. Default is 1048576 | ||
| Enable Dictionary | A boolean that indicates whether to include the dictionary compression strategy in the generated parquet document. Enabling dictionary allows for building a dictionary of values encountered in columns. | ||
| Validating | Boolean that enables schema validation when writing. | ||
| Writer Version | Parquet format version to use when writing. 1.0 ensures compatibility with older readers. Default is v1. The option is to set v2 |