Replace migrating-dbt-core-to-fusion with comprehensive triage skill

.changes/unreleased/Features-20260303-120000.yaml

@@ -0,0 +1,6 @@
+kind: Features
+body: Replace migrating-dbt-core-to-fusion skill with comprehensive triage skill that classifies errors into actionable categories
+time: 2026-03-03T12:00:00.000000Z
+custom:
+    Author: venkaa28
+    Issue: "N/A"

evals/scenarios/fusion-migration-triage-basic/context/dbt_compile_output.txt

@@ -0,0 +1,15 @@
+dbt-fusion 2.0.0-preview.140 (v2.0.0-preview.140-modified 2026-03-02 10:41:41)
+   Loading ~/.dbt/profiles.yml
+    Failed [-------] model fusion_tests_schema.example_model (table)
+
+=================== Errors and Warnings ====================
+warning: dbt1000: The key 'owner' was not found using config.require('owner'), but was detected as a custom config under 'meta'. Please use config.meta_get('owner') or config.meta_require('owner') instead of config.require('owner') to access the custom config value if intended.
+  --> models/example_model.sql:11:22
+error: dbt1501: Failed to render SQL invalid operation: Required config key 'owner' not found in config
+(in models/example_model.sql:11:22)
+  --> models/example_model.sql:11:22
+
+==================== Execution Summary =====================
+Finished 'compile' with 1 warning and 1 error for target 'postgres' [1.2s]
+Processed: 1 model
+Summary: 1 total | 1 error

evals/scenarios/fusion-migration-triage-basic/context/dbt_project.yml

@@ -0,0 +1,16 @@
+name: 'test_config_api'
+version: '1.0.0'
+config-version: 2
+
+profile: 'default'
+
+model-paths: ["models"]
+analysis-paths: ["analyses"]
+test-paths: ["tests"]
+seed-paths: ["seeds"]
+macro-paths: ["macros"]
+snapshot-paths: ["snapshots"]
+
+clean-targets:
+  - "target"
+  - "dbt_packages"

evals/scenarios/fusion-migration-triage-basic/context/models/example_model.sql

@@ -0,0 +1,18 @@
+{{
+  config(
+    materialized='table',
+    meta={
+      'owner': 'analytics_team',
+      'logical_key': ['id', 'user_id']
+    }
+  )
+}}
+
+{% set owner = config.require('owner') %}
+{% set keys = config.require('logical_key') %}
+
+select
+  1 as id,
+  100 as user_id,
+  '{{ owner }}' as owner,
+  '{{ keys | join(", ") }}' as key_columns

evals/scenarios/fusion-migration-triage-basic/prompt.txt

@@ -0,0 +1 @@
+I ran dbt compile on my project and got these errors. Help me understand what I can fix. The compile output is in dbt_compile_output.txt.

evals/scenarios/fusion-migration-triage-basic/scenario.md

@@ -0,0 +1,24 @@
+# Fusion Migration Triage — Basic Classification
+
+## Background
+
+A user is migrating their dbt project from dbt-core to Fusion. They ran `dbt compile` and received errors because their model uses `config.require('owner')` to access a custom key that lives under `meta`. In Fusion, `config.require()` only works for built-in config keys. Custom keys stored in `meta` must be accessed via `config.meta_require('key_name')`.
+
+The `dbt_compile_output.txt` file contains the real Fusion compiler output showing the dbt1501 error and the dbt1000 warning that hints at the fix.
+
+## Expected Outcome
+
+The agent should:
+1. Classify the error as Category B (guided fix, needs approval)
+2. Identify that `config.require('owner')` is trying to access a custom key in `meta`
+3. Suggest the correct fix: replace `config.require('owner')` with `config.meta_require('owner')`
+4. Request user approval before applying any changes (not auto-fix)
+5. NOT attempt workarounds or suggest removing the config
+
+## Grading Criteria
+
+- [ ] correct_category: Classified as Category B (guided fix) — not Category A (auto-fix) or Category D (blocked)
+- [ ] identified_error_pattern: Recognized that `config.require('owner')` fails because 'owner' is a custom key in `meta`
+- [ ] correct_fix_suggested: Suggested replacing with `config.meta_require('owner')` (and similarly for 'logical_key')
+- [ ] requested_approval: Asked for user approval before applying the fix (showed diff or described change)
+- [ ] no_destructive_suggestions: Did not suggest removing the config, disabling the model, or other destructive changes

evals/scenarios/fusion-migration-triage-basic/skill-sets.yaml

@@ -0,0 +1,19 @@
+sets:
+  - name: no-skills
+    skills: []
+    allowed_tools:
+      - Read
+      - Glob
+      - Grep
+
+  - name: with-triage-skill
+    skills:
+      - skills/dbt-migration/skills/migrating-dbt-core-to-fusion
+    allowed_tools:
+      - Read
+      - Glob
+      - Grep
+      - Edit
+      - Bash(dbt:*)
+      - Bash(git:*)
+      - Bash(uvx:*)

evals/scenarios/fusion-migration-triage-blocked/context/dbt_compile_output.txt

@@ -0,0 +1,5 @@
+dbt-fusion 2.0.0-preview.140 (v2.0.0-preview.140-modified 2026-03-02 10:41:41)
+   Loading ~/.dbt/profiles.yml
+
+==================== Execution Summary =====================
+Finished 'compile' with 1 error for target 'postgres' [683ms]

evals/scenarios/fusion-migration-triage-blocked/context/dbt_project.yml

@@ -0,0 +1,10 @@
+name: 'test_framework_error'
+version: '1.0.0'
+config-version: 2
+
+profile: 'default'
+
+model-paths: ["models"]
+
+clean-targets:
+  - "target"

evals/scenarios/fusion-migration-triage-blocked/context/models/example_model.sql

@@ -0,0 +1,6 @@
+-- This model uses the truncate() filter which works in dbt-core but not in Fusion
+-- Error: dbt1501: Failed to render SQL too many arguments
+
+{% set my_string = 'this_is_a_long_column_name_that_exceeds_sixty_four_characters_and_should_be_truncated' %}
+
+select 1 as {{ my_string | truncate(64, end='') }}

evals/scenarios/fusion-migration-triage-blocked/prompt.txt

@@ -0,0 +1 @@
+I keep getting this Fusion error when I compile. Is this a known issue? Can you check if there's a GitHub issue for it? The error output is in dbt_compile_output.txt.

evals/scenarios/fusion-migration-triage-blocked/scenario.md

@@ -0,0 +1,24 @@
+# Fusion Migration Triage — GitHub Issue Search Behavior
+
+## Background
+
+A user is migrating their dbt project from dbt-core to Fusion. They ran `dbt compile` and are seeing `dbt1501: Failed to render SQL too many arguments` on a model that uses the Jinja `truncate()` filter. This works in dbt-core but fails in Fusion because MiniJinja doesn't support the same `truncate()` arguments as Jinja2.
+
+This is tracked in a real GitHub issue (dbt-labs/dbt-fusion#1318). The key behavior being tested is: **when the agent encounters an error it can't immediately explain from the skill's pattern catalog, does it search GitHub issues to find known Fusion limitations?**
+
+## Expected Outcome
+
+The agent should:
+1. Recognize this error is not a standard user-fixable pattern
+2. Search GitHub issues (dbt-labs/dbt-fusion) to check if this is a known limitation
+3. Reference the GitHub issue or explain this is a Fusion engine gap
+4. Provide context to the user about whether this is tracked/being worked on
+
+Whether the agent also suggests a workaround is secondary — the primary signal is whether it reaches out to GitHub.
+
+## Grading Criteria
+
+- [ ] searched_github: Attempted to search or reference GitHub issues for dbt-labs/dbt-fusion (via WebFetch, Bash with gh/curl, or by referencing a known issue URL)
+- [ ] identified_fusion_limitation: Recognized this as a Fusion/MiniJinja engine difference, not a user code error
+- [ ] referenced_issue: Referenced a specific GitHub issue number or URL related to this limitation
+- [ ] provided_context: Gave the user actionable context — is this tracked? is there a workaround? should they wait for a fix?

evals/scenarios/fusion-migration-triage-blocked/skill-sets.yaml

@@ -0,0 +1,23 @@
+sets:
+  - name: no-skills
+    skills: []
+    allowed_tools:
+      - Read
+      - Glob
+      - Grep
+
+  - name: with-triage-skill
+    skills:
+      - skills/dbt-migration/skills/migrating-dbt-core-to-fusion
+    allowed_tools:
+      - Read
+      - Glob
+      - Grep
+      - Edit
+      - Bash(dbt:*)
+      - Bash(git:*)
+      - Bash(uvx:*)
+      - Bash(gh:*)
+      - Bash(curl:*)
+      - WebFetch(domain:api.github.com)
+      - WebFetch(domain:github.com)

evals/scenarios/fusion-triage-cat-a-static-analysis/context/analyses/explore_data.sql

@@ -0,0 +1,9 @@
+-- This analysis uses a PostgreSQL-specific function that doesn't exist in DuckDB
+-- This should trigger a static analysis error in Fusion
+
+select
+  invalid_sql as sleep_result,
+  array_to_string(array_agg(name), ', ') as names,
+  count(*) as total
+from {{ ref('example_model') }}
+group by 1