Skip to content

chat: use structured outputs for SearchBookContent MCP tool#1079

Draft
BenjaminMichaelis wants to merge 1 commit intomainfrom
agents/ai-chat-structured-output-review
Draft

chat: use structured outputs for SearchBookContent MCP tool#1079
BenjaminMichaelis wants to merge 1 commit intomainfrom
agents/ai-chat-structured-output-review

Conversation

@BenjaminMichaelis
Copy link
Copy Markdown
Member

Summary

Improves structured output consistency across the AI chat RAG pipeline by converting SearchBookContent — the primary semantic vector search tool — to use the same hybrid structured content pattern already established by FindBookHelpForDiagnostic and SearchListingsByCode.

The changes were reviewed and approved by Claude Opus 4.6 and GPT-5.5 subagents, iterating to agreement.

Changes

SearchBookContent converted to structured output

SearchBookContent was the only major RAG tool still returning plain text. It now returns a CallToolResult with:

  • UseStructuredContent = true and OutputSchemaType = typeof(SearchBookContentResult)
  • McpToolResultFormatter.GetModelInput() will now return the JSON StructuredContent to the model instead of the formatted text, consistent with all other structured tools
  • The human-readable text is preserved as the first Content block for MCP clients that display it

"No results" semantics fixed

The zero-results path previously returned McpToolResultFactory.CreateError() (IsError = true). Zero results from a search query is a valid outcome, not a tool failure. It now returns CreateHybridResult with an empty Matches: [] list, so the model always receives well-typed JSON conforming to the schema.

Validation failures (empty query, query too long, AI service not configured) remain correctly modeled as errors.

New structured result types

Added to McpToolResults.cs:

  • SearchBookContentMatchResult — one vector search hit (score, chapter number, heading, text chunk)
  • SearchBookContentResult — wrapper with Matches list; this is the advertised output schema

OutputSchemaType made explicit on all UseStructuredContent tools

Five tools that had UseStructuredContent = true but no OutputSchemaType were updated:
GetChapterList, GetChapterSections, GetDirectContentUrl, GetNavigationContext, GetChapterSummary.

The MCP SDK already auto-infers the output schema from the concrete return type (confirmed by pre-existing integration tests), so this is redundant but makes the intended schema contract explicit at the declaration site — consistent with how FindBookHelpForDiagnostic and SearchListingsByCode are declared.

Contract test updated

McpToolContractTests.McpToolsList_StructuredAndHybridTools_AdvertiseOutputSchema now asserts that search_book_content advertises an outputSchema, alongside the other structured tools.

What was NOT changed (and why)

  • Main chat response formatResponseTextFormat.CreateJsonSchemaFormat() does not exist in OpenAI SDK 2.7.0 for the Responses API. Schema-enforced output on the streaming text response isn't possible without switching to the Chat Completions API, and would break markdown rendering anyway. Both subagents agreed this is correct.
  • strictModeEnabled: true on tool input registration — Already correct in AIChatService.cs, untouched.
  • Plain-text tools (LookupConcept, CheckTopicCoverage, FindRelatedSections, etc.) — These return prose-heavy navigation summaries where structured content wouldn't add value for the model.

Testing

  • ✅ Solution builds: 0 errors, 0 warnings
  • ✅ All 11 chat unit tests pass
  • ✅ Contract test updated to cover search_book_content schema advertisement

- Convert SearchBookContent from Task<string> to Task<CallToolResult> with
  UseStructuredContent = true and OutputSchemaType = typeof(SearchBookContentResult),
  matching the pattern established by FindBookHelpForDiagnostic and SearchListingsByCode
- Add SearchBookContentMatchResult and SearchBookContentResult records to McpToolResults.cs
- Fix no-results path to return CreateHybridResult with empty Matches list instead of
  CreateError — zero search results is a valid outcome, not a tool failure
- Add explicit OutputSchemaType to GetChapterList, GetChapterSections, GetDirectContentUrl,
  GetNavigationContext, and GetChapterSummary for documentation clarity (MCP SDK already
  infers the output schema from the concrete return type, but explicit annotation makes
  the intended schema contract visible at the declaration site)
- Add search_book_content to McpToolContractTests outputSchema presence assertions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant