Anthropic structured output (Output.object / generateObject) broken — response_format ignored

[03-CiprianoG]

Problem

When using the Vercel AI SDK's Output.object({ schema }) with generateText() or generateObject() for Anthropic models (e.g., anthropic/claude-sonnet-4-6) through the Helicone gateway, the model returns prose/markdown instead of JSON, causing NoObjectGeneratedError: No object generated: could not parse the response.

Root Cause

The provider's buildRequestBody() converts the AI SDK's responseFormat to OpenAI's response_format parameter:

{
  "response_format": {
    "type": "json_schema",
    "json_schema": { "schema": { ... }, "strict": true, "name": "response" }
  }
}

The Helicone gateway passes this field through to Anthropic's API, but Anthropic does not recognize response_format — it's an OpenAI-specific parameter. Anthropic silently ignores it and the model generates unstructured text.

Reproduction

import { generateText, Output } from 'ai';
import { createHelicone } from '@helicone/ai-sdk-provider';
import { z } from 'zod';

const helicone = createHelicone({ apiKey: process.env.HELICONE_API_KEY });
const model = helicone('anthropic/claude-sonnet-4-6');

const { output } = await generateText({
  model,
  output: Output.object({
    schema: z.object({
      name: z.string(),
      age: z.number(),
      hobbies: z.array(z.string()),
    }),
  }),
  prompt: 'Generate a random person profile.',
});
// ❌ Throws: NoObjectGeneratedError: No object generated: could not parse the response.
// The model returned markdown like "# Person Profile\n**Name:** John..."

What's sent to the API (from Helicone request logs)

{
  "model": "anthropic/claude-sonnet-4-6",
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "schema": { "type": "object", "properties": { "name": { "type": "string" }, ... } },
      "strict": true,
      "name": "response"
    }
  },
  "messages": [...]
}

Anthropic ignores response_format entirely. The response is prose, not JSON.

What should be sent instead

Anthropic supports structured output natively via the output_format parameter:

{
  "model": "anthropic/claude-sonnet-4-6",
  "output_format": {
    "type": "json_schema",
    "json_schema": {
      "schema": { "type": "object", "properties": { "name": { "type": "string" }, ... } },
      "name": "response"
    }
  },
  "messages": [...]
}

Note: Anthropic's output_format does not use strict: true (that's OpenAI-specific).

Expected Behavior

Output.object({ schema }) and generateObject() should produce valid, schema-conforming JSON output when used with Anthropic models through the Helicone gateway.

Environment

• @helicone/ai-sdk-provider: 1.0.12
• ai (Vercel AI SDK): 5.x
• Models affected: All anthropic/* models
• Models NOT affected: openai/*, google/* (they use response_format correctly)

[03-CiprianoG]

Update: Root cause is in the AI Gateway, not the SDK provider

After investigation, the real fix needs to happen in Helicone/helicone (the AI Gateway), not in the SDK provider.

The gateway's toAnthropic() request translator ignores response_format entirely — it needs to translate it to Anthropic's native output_config.format.

Gateway issue: Helicone/helicone#5639
Gateway PR: Helicone/helicone#5640

The SDK provider PR (#26) has been closed as it was a workaround (trying to send output_format directly, which the gateway rejects as an unrecognized key).

Current workaround

Until the gateway fix is merged, the SDK provider can work around this by converting responseFormat into a json tool with tool_choice: required for Anthropic models — similar to how @ai-sdk/anthropic implements its jsonTool structured output mode. This forces Claude to respond via tool call with valid JSON matching the schema.