-
Notifications
You must be signed in to change notification settings - Fork 151
Description
Is this your first time submitting a feature request?
- I have searched the existing issues, and I could not find an existing issue for this feature
- I am requesting a straightforward feature that benefits all adapters
Describe the feature
Add optional configuration to append unique suffixes to test failure table names when using --store-failures
. This enables parallel test execution and preserves historical test failure data.
Current Problem
When running dbt test --store-failures
in parallel (e.g., hourly/daily Airflow DAGs), test failure tables overwrite each other because they use static names. This causes:
- Loss of historical audit data
- Conflicts in parallel execution
- Inability to debug intermittent failures
Proposed Solution
Add optional configuration in the base test materialization:
tests:
+store_failures: true
+store_failures_unique: true # Enable unique names (default: false)
+store_failures_suffix: 'hour' # Strategy: invocation_id, timestamp, date, hour, custom
This creates tables like:
test_not_null_orders_id_20240819_14
(hour suffix)test_not_null_orders_id_a1b2c3d4
(invocation_id suffix)
Implementation Location
dbt-adapters/src/dbt/include/global_project/macros/materializations/tests/test.sql
This is in the base adapter code, so ALL database adapters benefit automatically.
Who will this benefit?
- Teams running parallel test executions
- Organizations needing audit trails for compliance
- Anyone debugging intermittent test failures
- Production data pipelines with scheduled test runs
Backward Compatibility
Fully backward compatible - it's opt-in with store_failures_unique
defaulting to false
.
Are you interested in contributing this feature?
Yes, I have the implementation ready and will submit a PR following the contributing guide.