POC (testing) -- test with Parquet Metadata Cache enabled #16988

alamb · 2025-07-30T20:36:46Z

This PR is to test the performance of this PR from @nuno-faria

feat: Cache Parquet metadata in built in parquet reader #16971

When the parquet metadata cache is enabled

I do not intend to merge this PR

alamb · 2025-07-30T20:54:06Z

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.11.0-1016-gcp #16~24.04.1-Ubuntu SMP Wed May 28 02:40:52 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/default_to_on (5a824b1) to aab44fd diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

alamb · 2025-07-30T21:34:33Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_default_to_on
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_default_to_on ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │  1942.14 ms │          1911.39 ms │     no change │
│ QQuery 1     │   737.60 ms │           665.79 ms │ +1.11x faster │
│ QQuery 2     │  1417.91 ms │          1392.87 ms │     no change │
│ QQuery 3     │   673.20 ms │           646.67 ms │     no change │
│ QQuery 4     │  1369.56 ms │          1311.93 ms │     no change │
│ QQuery 5     │ 14334.89 ms │         13880.74 ms │     no change │
│ QQuery 6     │  2032.67 ms │          1996.48 ms │     no change │
│ QQuery 7     │  1919.91 ms │          1937.69 ms │     no change │
└──────────────┴─────────────┴─────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                  ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                  │ 24427.86ms │
│ Total Time (alamb_default_to_on)   │ 23743.56ms │
│ Average Time (HEAD)                │  3053.48ms │
│ Average Time (alamb_default_to_on) │  2967.94ms │
│ Queries Faster                     │          1 │
│ Queries Slower                     │          0 │
│ Queries with No Change             │          7 │
│ Queries with Failure               │          0 │
└────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_default_to_on ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.29 ms │             2.40 ms │     no change │
│ QQuery 1     │    34.32 ms │            28.63 ms │ +1.20x faster │
│ QQuery 2     │    82.60 ms │            73.41 ms │ +1.13x faster │
│ QQuery 3     │    98.83 ms │            89.38 ms │ +1.11x faster │
│ QQuery 4     │   632.89 ms │           584.87 ms │ +1.08x faster │
│ QQuery 5     │   872.35 ms │           875.69 ms │     no change │
│ QQuery 6     │     2.29 ms │             2.26 ms │     no change │
│ QQuery 7     │    38.56 ms │            33.38 ms │ +1.16x faster │
│ QQuery 8     │   861.20 ms │           849.56 ms │     no change │
│ QQuery 9     │  1199.66 ms │          1165.61 ms │     no change │
│ QQuery 10    │   264.93 ms │           245.34 ms │ +1.08x faster │
│ QQuery 11    │   293.89 ms │           276.05 ms │ +1.06x faster │
│ QQuery 12    │   876.57 ms │           874.01 ms │     no change │
│ QQuery 13    │  1245.28 ms │          1087.92 ms │ +1.14x faster │
│ QQuery 14    │   809.24 ms │           798.99 ms │     no change │
│ QQuery 15    │   782.99 ms │           783.56 ms │     no change │
│ QQuery 16    │  1633.51 ms │          1613.28 ms │     no change │
│ QQuery 17    │  1618.84 ms │          1606.02 ms │     no change │
│ QQuery 18    │  2899.33 ms │          2877.22 ms │     no change │
│ QQuery 19    │    86.36 ms │            80.57 ms │ +1.07x faster │
│ QQuery 20    │  1146.47 ms │          1153.81 ms │     no change │
│ QQuery 21    │  1277.78 ms │          1291.42 ms │     no change │
│ QQuery 22    │  2089.27 ms │          2151.21 ms │     no change │
│ QQuery 23    │  7495.04 ms │          7476.17 ms │     no change │
│ QQuery 24    │   441.25 ms │           424.51 ms │     no change │
│ QQuery 25    │   304.65 ms │           303.16 ms │     no change │
│ QQuery 26    │   440.59 ms │           421.18 ms │     no change │
│ QQuery 27    │  1530.60 ms │          1553.62 ms │     no change │
│ QQuery 28    │ 11823.51 ms │         11883.80 ms │     no change │
│ QQuery 29    │   529.94 ms │           535.79 ms │     no change │
│ QQuery 30    │   782.79 ms │           773.91 ms │     no change │
│ QQuery 31    │   807.47 ms │           795.37 ms │     no change │
│ QQuery 32    │  2416.28 ms │          2445.13 ms │     no change │
│ QQuery 33    │  3174.35 ms │          3177.08 ms │     no change │
│ QQuery 34    │  3204.78 ms │          3203.55 ms │     no change │
│ QQuery 35    │  1287.45 ms │          1239.27 ms │     no change │
│ QQuery 36    │   122.28 ms │           123.83 ms │     no change │
│ QQuery 37    │    52.67 ms │            54.51 ms │     no change │
│ QQuery 38    │   115.34 ms │           119.25 ms │     no change │
│ QQuery 39    │   197.55 ms │           194.75 ms │     no change │
│ QQuery 40    │    44.35 ms │            40.65 ms │ +1.09x faster │
│ QQuery 41    │    38.39 ms │            38.56 ms │     no change │
│ QQuery 42    │    31.89 ms │            32.78 ms │     no change │
└──────────────┴─────────────┴─────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                  ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                  │ 53690.62ms │
│ Total Time (alamb_default_to_on)   │ 53381.52ms │
│ Average Time (HEAD)                │  1248.62ms │
│ Average Time (alamb_default_to_on) │  1241.43ms │
│ Queries Faster                     │         10 │
│ Queries Slower                     │          0 │
│ Queries with No Change             │         33 │
│ Queries with Failure               │          0 │
└────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ alamb_default_to_on ┃    Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ QQuery 1     │  97.16 ms │            96.90 ms │ no change │
│ QQuery 2     │  21.16 ms │            20.62 ms │ no change │
│ QQuery 3     │  32.88 ms │            33.30 ms │ no change │
│ QQuery 4     │  18.63 ms │            18.73 ms │ no change │
│ QQuery 5     │  49.36 ms │            49.24 ms │ no change │
│ QQuery 6     │  11.90 ms │            11.90 ms │ no change │
│ QQuery 7     │  86.13 ms │            87.34 ms │ no change │
│ QQuery 8     │  24.94 ms │            24.49 ms │ no change │
│ QQuery 9     │  53.97 ms │            53.28 ms │ no change │
│ QQuery 10    │  43.55 ms │            43.40 ms │ no change │
│ QQuery 11    │  11.19 ms │            11.15 ms │ no change │
│ QQuery 12    │  35.01 ms │            35.03 ms │ no change │
│ QQuery 13    │  26.50 ms │            26.74 ms │ no change │
│ QQuery 14    │   9.63 ms │             9.69 ms │ no change │
│ QQuery 15    │  18.91 ms │            19.19 ms │ no change │
│ QQuery 16    │  18.20 ms │            18.31 ms │ no change │
│ QQuery 17    │  97.29 ms │            97.81 ms │ no change │
│ QQuery 18    │ 193.29 ms │           195.43 ms │ no change │
│ QQuery 19    │  25.55 ms │            24.55 ms │ no change │
│ QQuery 20    │  30.76 ms │            30.84 ms │ no change │
│ QQuery 21    │ 144.81 ms │           144.80 ms │ no change │
│ QQuery 22    │  14.63 ms │            15.27 ms │ no change │
└──────────────┴───────────┴─────────────────────┴───────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                  ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                  │ 1065.45ms │
│ Total Time (alamb_default_to_on)   │ 1068.01ms │
│ Average Time (HEAD)                │   48.43ms │
│ Average Time (alamb_default_to_on) │   48.55ms │
│ Queries Faster                     │         0 │
│ Queries Slower                     │         0 │
│ Queries with No Change             │        22 │
│ Queries with Failure               │         0 │
└────────────────────────────────────┴───────────┘

alamb · 2025-07-31T11:47:31Z

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.11.0-1016-gcp #16~24.04.1-Ubuntu SMP Wed May 28 02:40:52 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/default_to_on (5a824b1) to aab44fd diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

alamb · 2025-07-31T12:19:56Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_default_to_on
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_default_to_on ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │  2035.06 ms │          2078.14 ms │     no change │
│ QQuery 1     │   741.78 ms │           676.55 ms │ +1.10x faster │
│ QQuery 2     │  1436.51 ms │          1397.33 ms │     no change │
│ QQuery 3     │   674.94 ms │           641.75 ms │     no change │
│ QQuery 4     │  1372.57 ms │          1297.93 ms │ +1.06x faster │
│ QQuery 5     │ 14473.26 ms │         14100.05 ms │     no change │
│ QQuery 6     │  2036.22 ms │          2019.89 ms │     no change │
│ QQuery 7     │  1863.23 ms │          1913.17 ms │     no change │
└──────────────┴─────────────┴─────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                  ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                  │ 24633.57ms │
│ Total Time (alamb_default_to_on)   │ 24124.81ms │
│ Average Time (HEAD)                │  3079.20ms │
│ Average Time (alamb_default_to_on) │  3015.60ms │
│ Queries Faster                     │          2 │
│ Queries Slower                     │          0 │
│ Queries with No Change             │          6 │
│ Queries with Failure               │          0 │
└────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_default_to_on ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.26 ms │             2.22 ms │     no change │
│ QQuery 1     │    33.89 ms │            28.60 ms │ +1.19x faster │
│ QQuery 2     │    83.44 ms │            74.02 ms │ +1.13x faster │
│ QQuery 3     │   101.99 ms │            93.20 ms │ +1.09x faster │
│ QQuery 4     │   615.06 ms │           593.42 ms │     no change │
│ QQuery 5     │   848.97 ms │           869.94 ms │     no change │
│ QQuery 6     │     2.44 ms │             2.24 ms │ +1.09x faster │
│ QQuery 7     │    39.42 ms │            32.72 ms │ +1.20x faster │
│ QQuery 8     │   888.32 ms │           866.81 ms │     no change │
│ QQuery 9     │  1205.97 ms │          1193.89 ms │     no change │
│ QQuery 10    │   269.43 ms │           246.06 ms │ +1.09x faster │
│ QQuery 11    │   296.54 ms │           278.49 ms │ +1.06x faster │
│ QQuery 12    │   881.97 ms │           872.82 ms │     no change │
│ QQuery 13    │  1278.77 ms │          1256.67 ms │     no change │
│ QQuery 14    │   823.22 ms │           827.70 ms │     no change │
│ QQuery 15    │   817.89 ms │           799.20 ms │     no change │
│ QQuery 16    │  1636.15 ms │          1628.92 ms │     no change │
│ QQuery 17    │  1622.81 ms │          1614.52 ms │     no change │
│ QQuery 18    │  2839.14 ms │          2895.62 ms │     no change │
│ QQuery 19    │    86.07 ms │            78.77 ms │ +1.09x faster │
│ QQuery 20    │  1121.67 ms │          1160.45 ms │     no change │
│ QQuery 21    │  1303.22 ms │          1310.97 ms │     no change │
│ QQuery 22    │  2128.95 ms │          2179.15 ms │     no change │
│ QQuery 23    │  7507.99 ms │          7467.17 ms │     no change │
│ QQuery 24    │   449.13 ms │           421.67 ms │ +1.07x faster │
│ QQuery 25    │   296.80 ms │           299.42 ms │     no change │
│ QQuery 26    │   439.43 ms │           421.41 ms │     no change │
│ QQuery 27    │  1580.09 ms │          1562.90 ms │     no change │
│ QQuery 28    │ 12036.37 ms │         12005.34 ms │     no change │
│ QQuery 29    │   533.07 ms │           515.63 ms │     no change │
│ QQuery 30    │   770.43 ms │           792.33 ms │     no change │
│ QQuery 31    │   785.81 ms │           786.36 ms │     no change │
│ QQuery 32    │  2394.77 ms │          2464.91 ms │     no change │
│ QQuery 33    │  3096.85 ms │          3199.64 ms │     no change │
│ QQuery 34    │  3132.99 ms │          3188.48 ms │     no change │
│ QQuery 35    │  1240.88 ms │          1267.96 ms │     no change │
│ QQuery 36    │   114.91 ms │           122.49 ms │  1.07x slower │
│ QQuery 37    │    53.60 ms │            51.65 ms │     no change │
│ QQuery 38    │   116.43 ms │           120.20 ms │     no change │
│ QQuery 39    │   193.84 ms │           198.11 ms │     no change │
│ QQuery 40    │    40.13 ms │            42.24 ms │  1.05x slower │
│ QQuery 41    │    37.88 ms │            39.59 ms │     no change │
│ QQuery 42    │    33.34 ms │            33.31 ms │     no change │
└──────────────┴─────────────┴─────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                  ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                  │ 53782.37ms │
│ Total Time (alamb_default_to_on)   │ 53907.21ms │
│ Average Time (HEAD)                │  1250.75ms │
│ Average Time (alamb_default_to_on) │  1253.66ms │
│ Queries Faster                     │          9 │
│ Queries Slower                     │          2 │
│ Queries with No Change             │         32 │
│ Queries with Failure               │          0 │
└────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ alamb_default_to_on ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │  95.53 ms │            98.41 ms │     no change │
│ QQuery 2     │  20.89 ms │            20.85 ms │     no change │
│ QQuery 3     │  32.25 ms │            32.18 ms │     no change │
│ QQuery 4     │  18.67 ms │            18.89 ms │     no change │
│ QQuery 5     │  49.75 ms │            49.32 ms │     no change │
│ QQuery 6     │  11.94 ms │            11.96 ms │     no change │
│ QQuery 7     │  91.78 ms │            88.38 ms │     no change │
│ QQuery 8     │  25.43 ms │            25.28 ms │     no change │
│ QQuery 9     │  54.36 ms │            53.76 ms │     no change │
│ QQuery 10    │  44.57 ms │            43.29 ms │     no change │
│ QQuery 11    │  11.17 ms │            11.01 ms │     no change │
│ QQuery 12    │  35.44 ms │            35.01 ms │     no change │
│ QQuery 13    │  27.08 ms │            26.82 ms │     no change │
│ QQuery 14    │   9.81 ms │            10.01 ms │     no change │
│ QQuery 15    │  19.83 ms │            19.76 ms │     no change │
│ QQuery 16    │  18.50 ms │            18.10 ms │     no change │
│ QQuery 17    │  97.63 ms │            96.30 ms │     no change │
│ QQuery 18    │ 202.30 ms │           189.60 ms │ +1.07x faster │
│ QQuery 19    │  25.93 ms │            24.69 ms │     no change │
│ QQuery 20    │  31.38 ms │            31.90 ms │     no change │
│ QQuery 21    │ 146.28 ms │           146.95 ms │     no change │
│ QQuery 22    │  15.46 ms │            14.88 ms │     no change │
└──────────────┴───────────┴─────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                  ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                  │ 1085.99ms │
│ Total Time (alamb_default_to_on)   │ 1067.36ms │
│ Average Time (HEAD)                │   49.36ms │
│ Average Time (alamb_default_to_on) │   48.52ms │
│ Queries Faster                     │         1 │
│ Queries Slower                     │         0 │
│ Queries with No Change             │        21 │
│ Queries with Failure               │         0 │
└────────────────────────────────────┴───────────┘

alamb · 2025-07-31T21:16:03Z

These are some pretty consistent and good improvements. Love it. Let's focus on getting

feat: Cache Parquet metadata in built in parquet reader #16971
in and merged and on by default

nuno-faria · 2025-07-31T22:46:05Z

@alamb Would it make sense to have a benchmark to test small/point queries? I would be happy to contribute in case it makes sense.

alamb · 2025-08-01T13:55:35Z

@alamb Would it make sense to have a benchmark to test small/point queries? I would be happy to contribute in case it makes sense.

I think it would make sense if it covered something different than ClickBench

clickbench_partitioned already covers several point queries (like selects 10k rows out of 100M rows) -- is that what you mean?

In terms of actual single row lookups, I am not sure about that benchmark it might end up being pretty specialized and hard to manage (the variability of benchmarks is already a problem for us).

-- interestingly @zhuqi-lucas @JigaoLui and I were just discussing this earlier today: #17010 (comment)

nuno-faria and others added 2 commits July 29, 2025 19:03

feat: Cache Parquet metadata

1452333

Default cache_metadata to on

5a824b1

alamb mentioned this pull request Jul 30, 2025

feat: Cache Parquet metadata in built in parquet reader #16971

Merged

alamb closed this Jul 31, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

POC (testing) -- test with Parquet Metadata Cache enabled #16988

POC (testing) -- test with Parquet Metadata Cache enabled #16988

Uh oh!

alamb commented Jul 30, 2025

Uh oh!

alamb commented Jul 30, 2025

Uh oh!

alamb commented Jul 30, 2025

Uh oh!

alamb commented Jul 31, 2025

Uh oh!

alamb commented Jul 31, 2025

Uh oh!

alamb commented Jul 31, 2025

Uh oh!

nuno-faria commented Jul 31, 2025

Uh oh!

alamb commented Aug 1, 2025

Uh oh!

Uh oh!

POC (testing) -- test with Parquet Metadata Cache enabled #16988

POC (testing) -- test with Parquet Metadata Cache enabled #16988

Uh oh!

Conversation

alamb commented Jul 30, 2025

Uh oh!

alamb commented Jul 30, 2025

Uh oh!

alamb commented Jul 30, 2025

Uh oh!

alamb commented Jul 31, 2025

Uh oh!

alamb commented Jul 31, 2025

Uh oh!

alamb commented Jul 31, 2025

Uh oh!

nuno-faria commented Jul 31, 2025

Uh oh!

alamb commented Aug 1, 2025

Uh oh!

Uh oh!