Structured Output with gpt-oss-20b

[facundofleitas]

The 'with_structured_output'method (ChatBedrockConverse) is not working for gpt-oss-20b

The ToolsOutputParser returns None; the code works for other models on AWS Bedrock, such as Haiku.

[michaelnchin]

Hi @facundofleitas , thank you for submitting an issue - could you provide a minimal sample to reproduce this behavior?

[facundofleitas]

from pydantic import BaseModel, Field
from langchain_aws import ChatBedrockConverse

class OutputExample(BaseModel):
    response: str = Field(description="The response from the model")

model = ChatBedrockConverse(
    model_id="openai.gpt-oss-20b-1:0",
    temperature=0.0,
    region_name="us-west-2"
)

model_with_output = model.with_structured_output(OutputExample)
response = model_with_output.invoke("What is the capital of Uruguay?")
print(response)

Hi! Thanks for the reply!

The response is 'None'.

[michaelnchin]

The Bedrock-hosted gpt-oss models don't fully support tool calling at this time. This is being worked on internally, I'll update here when we have more to share.

For now, I would recommend running gpt-oss via another provider or locally (i.e. on Ollama) if you need to use tools with these models.

[ellisonbg]

@michaelnchin do we have an external issue we can point people to for status of this?

[michaelnchin]

Not at the moment - the only official communication has been via the initial AWS blogpost, which does mention tool calling support: https://aws.amazon.com/blogs/aws/openai-open-weight-models-now-available-on-aws/

Bedrock's Converse API model feature support page does not yet include entries for the gpt-oss models, I think this will be updated once the status of tool (and streaming) support is clearer.

[facundofleitas]

Hello, any update?

[michaelnchin]

Hi @facundofleitas, tool use should now be working on both gpt-oss models with ChatBedrockConverse. Let us know if you are still having any issues.

In regards to structured output: gpt-oss does not currently support forced tool calling on Bedrock, so it is possible that the model may opt not to execute a tool call (ex. if the model deems it unnecessary for a simple task), even when invoked using .with_structured_output. Invocations with with_structured_output that return None are indicative of such cases, where there are no tool calls available to be parsed from the response.

You may have higher success rates using with_structured_output with gpt-oss on more complex schemas - for example, this should work consistently:

from typing import Optional
from pydantic import BaseModel, Field
from langchain_aws import ChatBedrockConverse, ChatBedrock

class Joke(BaseModel):
    """Joke to tell user."""

    setup: str = Field(description="The setup of the joke")
    punchline: str = Field(description="The punchline to the joke")
    rating: Optional[int] = Field(
        default=None, description="How funny the joke is, from 1 to 10"
    )

model = ChatBedrockConverse(
    model_id="openai.gpt-oss-20b-1:0",
    region_name="us-west-2",
)

model_with_output = model.with_structured_output(Joke)
response = model_with_output.invoke("Tell me a funny joke.")
print(response)

Output:

setup='Why did the scarecrow win an award?' punchline='Because he was outstanding in his field!' rating=8

[carl-krikorian]

Apologies for being a bit late to this but I am using langchain-aws 0.2.31 getting a lot of issues with gpt-oss.
Had to override the _stream function using boto3 converse API for it to work.
But now the bind_tools method is throwing the following error.

    llm_with_tools = llm.bind_tools([SubmitExplanation, MyTool], tool_choice="any")
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "xxxxxx/lib/python3.11/site-packages/langchain_aws/chat_models/bedrock_converse.py", line 903, in bind_tools
    raise ValueError(
ValueError: Model openai.gpt-oss-20b-1:0 does not currently support tool_choice. Please see https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ToolChoice.html for the latest documentation on models that support tool choice.

However, from the above conversation and from the document catalog it should be able to:

This is my code slightly modified, please advise

def build_chain(llm):
    class SubmitExplanation(BaseModel):
        """Submit the an explanation as to why you can't fulfill the request."""

        explanation: str = Field(
            ..., description="The explanation for why you can't fulfill the request"
        )

    class MyTool(BaseModel):
        """Submit tool to do stuff"""

        output: str = Field(..., description="Your answer")

    llm_with_tools = llm.bind_tools([SubmitExplanation, MyTool], tool_choice="any")
    my_prompt = ChatPromptTemplate.from_messages(
        [
            ("system", "{system}"),
            ("placeholder", "{messages}"),
            ("user", "{input}"),
        ]
    )
    my_chain = my_prompt | llm_with_tools
    return my_chain

[michaelnchin]

Hi @carl-krikorian, Bedrock doesn't support forced tool calling for gpt-oss models, so we've disabled use of tool_choice for these models. bind_tools and tool calling should still work otherwise, though.

Had to override the _stream function using boto3 converse API for it to work.

Streaming is now supported for gpt-oss + ChatBedrockConverse, but it isn't enabled by default yet - we have a pending fix that will be in tomorrow's langchain-aws==0.2.32 release. But for now, you should set disable_streaming==False manually:

model = ChatBedrockConverse(
    model_id="openai.gpt-oss-120b-1:0",
    region_name="us-west-2",
    disable_streaming=False
)