Create in-memory index ?

[bubbajoe]

Hello, this might be a stupid question but is it possible to create an in-memory index on a table?

Obviously the purpose would be to improve searches / ordering. Thanks

[jaylmiller]

You can implement your own TableProvider to do this. I don't believe this is built in to any of the TableProviders that ship with DataFusion. Checkout https://github.com/apache/arrow-datafusion/blob/main/datafusion-examples/examples/custom_datasource.rs

[Jefffrey]

Also another discussion with in-depth discussion on indexing: #9963