Skip to content

Commit 01b608b

Browse files
committed
Update config to match new parquet writer
1 parent e3a6fc1 commit 01b608b

File tree

2 files changed

+4
-4
lines changed

2 files changed

+4
-4
lines changed

datafusion/common/src/config.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -604,9 +604,9 @@ config_namespace! {
604604
/// (writing) Sets column index truncate length
605605
pub column_index_truncate_length: Option<usize>, default = Some(64)
606606

607-
/// (writing) Sets statictics truncate length. If NULL, uses
607+
/// (writing) Sets statistics truncate length. If NULL, uses
608608
/// default parquet writer setting
609-
pub statistics_truncate_length: Option<usize>, default = None
609+
pub statistics_truncate_length: Option<usize>, default = Some(64)
610610

611611
/// (writing) Sets best effort maximum number of rows in data page
612612
pub data_page_row_count_limit: usize, default = 20_000

datafusion/sqllogictest/test_files/information_schema.slt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -252,7 +252,7 @@ datafusion.execution.parquet.schema_force_view_types true
252252
datafusion.execution.parquet.skip_arrow_metadata false
253253
datafusion.execution.parquet.skip_metadata true
254254
datafusion.execution.parquet.statistics_enabled page
255-
datafusion.execution.parquet.statistics_truncate_length NULL
255+
datafusion.execution.parquet.statistics_truncate_length 64
256256
datafusion.execution.parquet.write_batch_size 1024
257257
datafusion.execution.parquet.writer_version 1.0
258258
datafusion.execution.planning_concurrency 13
@@ -363,7 +363,7 @@ datafusion.execution.parquet.schema_force_view_types true (reading) If true, par
363363
datafusion.execution.parquet.skip_arrow_metadata false (writing) Skip encoding the embedded arrow metadata in the KV_meta This is analogous to the `ArrowWriterOptions::with_skip_arrow_metadata`. Refer to <https://docs.rs/parquet/53.3.0/parquet/arrow/arrow_writer/struct.ArrowWriterOptions.html#method.with_skip_arrow_metadata>
364364
datafusion.execution.parquet.skip_metadata true (reading) If true, the parquet reader skip the optional embedded metadata that may be in the file Schema. This setting can help avoid schema conflicts when querying multiple parquet files with schemas containing compatible types but different metadata
365365
datafusion.execution.parquet.statistics_enabled page (writing) Sets if statistics are enabled for any column Valid values are: "none", "chunk", and "page" These values are not case sensitive. If NULL, uses default parquet writer setting
366-
datafusion.execution.parquet.statistics_truncate_length NULL (writing) Sets statictics truncate length. If NULL, uses default parquet writer setting
366+
datafusion.execution.parquet.statistics_truncate_length 64 (writing) Sets statistics truncate length. If NULL, uses default parquet writer setting
367367
datafusion.execution.parquet.write_batch_size 1024 (writing) Sets write_batch_size in bytes
368368
datafusion.execution.parquet.writer_version 1.0 (writing) Sets parquet writer version valid values are "1.0" and "2.0"
369369
datafusion.execution.planning_concurrency 13 Fan-out during initial physical planning. This is mostly use to plan `UNION` children in parallel. Defaults to the number of CPU cores on the system

0 commit comments

Comments
 (0)