-
Notifications
You must be signed in to change notification settings - Fork 140
Closed as not planned
Labels
Description
Describe the feature
In Databricks you can recover partitions from existing parquet files when creating a table without partitions. When you define the partitions in the source (dbt) you also need to define the schema of the table. If you don't define a schema without partitions, the recover partitions still works, only the dbt_external_tables won't run the recover partitions part (cause no partitions).
Describe alternatives you've considered
Run it manually afterwards; Think a nice fix would be to supply it as a parameter or the ability to run it from cli afterwards.
Additional context
I asume this is only Spark / Databricks related.
Who will this benefit?
Anyone using parquet sources that are partitioned, but don't want to supply a schema in the source file (dbt).