Skip to content

Blog post about TopK filter pushdown #15513

@alamb

Description

@alamb

Is your feature request related to a problem or challenge?

@adriangb has a great PR to add dynamic filtering with TopK in DataFusion here:

Describe the solution you'd like

It would be really nice to write a blog post about this optimization in DataFusion (and maybe call out how it works)

Here are some related work

High level points

  • Study DuckDB plan Make ClickBench Q23 Go Faster #15177 (hos much faster)
  • Went back and forth to design something that could be general purpose
  • Did benchmarking,
  • Results are amazing, etc

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

documentationImprovements or additions to documentationenhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions