-
Notifications
You must be signed in to change notification settings - Fork 286
Open
Labels
triage:productIn Product's queueIn Product's queuetype:bugSomething isn't working as documentedSomething isn't working as documented
Description
Is this a new bug?
- I believe this is a new bug
- I have searched the existing issues, and I could not find an existing issue for this bug
Which packages are affected?
- dbt-adapters
- dbt-tests-adapter
- dbt-athena
- dbt-athena-community
- dbt-bigquery
- dbt-postgres
- dbt-redshift
- dbt-snowflake
- dbt-spark
Current Behavior
When create_table_as_with_partitions is invoked (e.g. after a TOO_MANY_OPEN_PARTITIONS fallback in safe_create_table_as) and the source query returns zero rows, get_partition_batches returns an empty list. The batch for loop is skipped entirely, so the target table is never created.
Any post-hook referencing the relation (e.g. OPTIMIZE or VACUUM) then fails:
TABLE_NOT_FOUND: Table ... does not exist
Expected Behavior
The target table should always be created after create_table_as_with_partitions completes, even when the source data is empty. An empty table is a valid and expected result for a zero-row run.
Steps To Reproduce
- Configure a partitioned Iceberg model with a post-hook:
config: materialized: incremental incremental_strategy: insert_overwrite partitioned_by: [date_col] post_hook: "OPTIMIZE {{ this }} REWRITE DATA USING BIN_PACK"
- Run the model when the source query returns zero rows (e.g. a date-filtered incremental run on an empty window).
- The run falls back to
create_table_as_with_partitionsviasafe_create_table_as. get_partition_batchesreturns[], the batch loop is skipped, and no target table is created.- The post-hook runs and fails with
TABLE_NOT_FOUND.
Relevant log output
Runtime Error
TABLE_NOT_FOUND: Table <schema>.<model> does not exist
> in post-hook: "OPTIMIZE <schema>.<model> REWRITE DATA USING BIN_PACK"Environment
- OS: macOS
- Python: 3.12.8
- dbt-adapters: 1.22.8
- dbt-athena: 1.9.4Additional Context
Root cause is in create_table_as_with_partitions (macros/materializations/models/table/create_table_as.sql):
{% set partitions_batches = get_partition_batches(sql=tmp_relation, as_subquery=False) %}
{%- for batch in partitions_batches -%}
{%- if loop.index == 1 -%}
{# CREATE TABLE AS SELECT ... WHERE batch — only runs if batches exist #}
{%- else -%}
{# INSERT INTO ... WHERE batch #}
{%- endif -%}
{%- endfor -%}
{# If partitions_batches is empty, the loop never runs and no table is created #}
{%- do drop_relation(tmp_relation) -%}Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
triage:productIn Product's queueIn Product's queuetype:bugSomething isn't working as documentedSomething isn't working as documented