Skip to content

Conversation

@andrzej-pomirski-yohana
Copy link

Many LLMs do not support references in the JSON Schemas for tool inputs/outputs and if they do, they mostly only support root-level references. Schemas produced with Zod to JSON Schema do not comply with this restriction thus, if using strict tool calling (e.g. in OpenAI), the request fails. While this is not documented as a limitation in the MCP Specification, for convenience we should avoid triggering this issue.

Motivation and Context

When trying out the TypeScript SDK we have used nested definitions for a group of fields. While I cannot provide the exact schema, but here's a quick mock:

export const locationSchema = z.object({
  testId: z.string().optional().describe("Test ID"),
  secondTestId: z.string().optional().describe("Second test ID")
});

export const searchParamsSchema = z.object({
  startLocation: locationSchema.describe("Test 1"),
  endLocation: locationSchema.describe("Test 2")
});

How Has This Been Tested?

I have tested my code with the lines from this PR and without it. Without my change, strict tool calling ends up in:

OpenAIException - {\n  "error": {\n    "message": "Invalid schema for function \'search\': In context=(\'properties\', \'endLocation\', \'properties\', \'testId\'), reference can only point to definitions defined at the top level of the schema.",\n    "type": "invalid_request_error",\n    "param": "tools[3].parameters",\n    "code": "invalid_function_parameters"\n  }

With my changes, the code runs properly.

Breaking Changes

No breaking changes and, to the best of my knowledge, no LLM providers charge extra for longer JSON Schemas, so this should not impact the users negatively.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

Additional context

Many LLMs do not support references in the JSON Schemas for tool inputs/outputs and if they do, they mostly only support root-level references. Schemas produced with Zod to JSON Schema do not comply with this restriction thus, if using strict tool calling (e.g. in OpenAI), the request fails.
@andrzej-pomirski-yohana andrzej-pomirski-yohana requested a review from a team as a code owner November 5, 2025 17:06
@mgyarmathy
Copy link

mgyarmathy commented Nov 5, 2025

This PR looks like it addresses similar concerns to #1080. Perhaps we close this in favor of that one?

@andrzej-pomirski-yohana
Copy link
Author

@mgyarmathy I'm fine either way as long as there's a way to remove the refs :)

@cliffhall
Copy link
Member

Please see the discussion in #1080.

To recap:

  1. fast-mcp, a prominent framework for developing mcp servers is leaning into $refs and has stated they no intention of letting up. Therefore this issue will continue, and LLMs will be forced to handle $refs.
  2. Clients and servers can resolve $refs before presenting to LLMs if need be.
  3. Most importantly, we use zod-to-json-schema right now, because we are on Zod 3. However Zod 4 has toJSONSchema and there's a good likelihood that we will move to Z4, but then we'd be faced with having to introduce a breaking change if we remove the duplicated dependency (zod-to-json-schema) for this one feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants