Skip to content

Add version awareness to product_docs toolset#638

Open
mirnawong1 wants to merge 4 commits intomainfrom
add/product-docs-version
Open

Add version awareness to product_docs toolset#638
mirnawong1 wants to merge 4 commits intomainfrom
add/product-docs-version

Conversation

@mirnawong1
Copy link
Contributor

Summary

Detect the user's dbt version from dbt_project.yml require-dbt-version at server startup and surface it in search/page responses so the LLM can give version-contextualized guidance. EOL pages (v1.6 and older) are annotated with a warning, and both tool prompts now include VERSION AWARENESS instructions for the LLM.

What Changed

Version detection at startup:

  • DbtProjectYaml now parses the require-dbt-version field from dbt_project.yml
  • New parse_dbt_version_minor() helper extracts the minor version (e.g. ">=1.8.0" becomes "1.8")
  • Config carries the detected dbt_version and passes it through to ProductDocsToolContext at registration time (one-time, no per-call overhead)

Version-aware responses:

  • SearchProductDocsResponse and GetProductDocPagesResponse now include a dbt_project_version field
  • ProductDocPageResponse has a new version_note field for EOL page flagging
  • detect_eol_page() identifies pages under the /core-upgrade/Older versions/ path
  • EOL pages get a >>> VERSION NOTICE warning prepended to their content

LLM prompt updates:

  • Both search_product_docs and get_product_doc_pages prompts now include a VERSION AWARENESS section instructing the LLM to:
    • State the user's detected dbt version upfront
    • Flag features that require a newer version than the user's
    • Warn about EOL content and suggest current docs instead
    • Never silently present content from a mismatched version

User impact: Zero-config. Version detection works automatically for users who already have DBT_PROJECT_DIR set and require-dbt-version in their dbt_project.yml. When version is unavailable, behavior is unchanged.

Why

Users reported that the product_docs toolset could return documentation for dbt versions that don't match their project -- including EOL versions (v1.6 and older) that are no longer supported. Since docs.getdbt.com serves a single unversioned corpus, the MCP server needs to provide version context so the LLM can guide users appropriately (e.g. flagging newer features they can't use yet, or warning about outdated EOL content).

Related Issues

Closes #
Related to #

Checklist

  • I have performed a self-review of my code
  • I have made corresponding changes to the documentation (in https://github.com/dbt-labs/docs.getdbt.com) if required -- No docs changes needed; this is internal MCP behavior
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Additional Notes

  • 13 new tests covering: version parsing (7 cases), EOL page detection (4 cases), version propagation in search/page responses (5 cases), and EOL page annotation (2 cases). Full suite: 451 tests pass.
  • Version detection is single-project only (DBT_PROJECT_DIR) and requires a server restart to pick up dbt_project.yml changes -- consistent with the existing MCP architecture.
  • EOL content is intentionally not filtered from search results. Instead, the LLM is instructed to clearly flag it, which serves as a gentle upgrade nudge.
  • Tested locally with the jaffle-shop project (require-dbt-version: ">=1.5.0") via MCP Inspector -- version detection, response fields, and EOL annotation all work as expected.


version_note: str | None = None
if detect_eol_page(normalized):
version_note = "This page is for an end-of-life dbt Core version (v1.6 or older)."
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honest question: would it be more reliable to determine EOL based on the parsed version number rather than the contents of the page?

dbt_version: str | None = None
project_yml = settings.dbt_project_yml
if project_yml and project_yml.require_dbt_version:
dbt_version = parse_dbt_version_minor(project_yml.require_dbt_version)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this hold the full dbt version or just the minor version?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the full version, so I would suggest renaming the function.

return None
raw = require_dbt_version if isinstance(require_dbt_version, str) else require_dbt_version[0]
match = _DBT_MINOR_VERSION_RE.search(raw)
return f"1.{match.group(1)}" if match else None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be a bit flaky if we ever release a 2.0 version. I don't think we should hard-code a 1. prefix.

Comment on lines +22 to +26
def parse_dbt_version_minor(require_dbt_version: str | list[str] | None) -> str | None:
"""Extract the minimum dbt minor version (e.g. ``"1.8"``) from a ``require-dbt-version`` spec.

Handles common forms: ``">=1.8.0"``, ``"1.8"``, ``[">=1.8.0", "<2.0"]``.
Returns ``None`` when the version cannot be determined.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we include an example of what this function returns in the docstring? Or, use a more specific type than string. For example:

@dataclass
class SemVer:
  major: int
  minor: int
  patch: str # pre-releases can contain non-numeric patch versions

Detect the user's dbt version from dbt_project.yml require-dbt-version
at server startup and surface it in search/page responses so the LLM
can give version-contextualized guidance. EOL pages (v1.6 and older)
are annotated with a warning, and both tool prompts now include
VERSION AWARENESS instructions for the LLM.

Made-with: Cursor
@jairus-m
Copy link
Collaborator

DbtProjectYaml now parses the require-dbt-version field from dbt_project.yml

Just some high-level thoughts on the above!

In the field and in my experience with past projects, require-dbt-version was rarely populated in the config, especially as it's an optional field. Also it does not reliability give the exact version which may be more helpful than a range that comes with a required version.

Would another approach be using the existing dbt CLI executable instead? So using the DBT_PATH and then quite literally running $DBT_PATH --version in a subprocess, then parsing that output? The complexity would be parsing the output of dbt core, dbt Cloud CLI, and dbt Fusion. I think there is an existing implementation in the code that already detects the binary type so maybe that could be extended or leveraged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants