[Bug] LiteLLM Ollama provider fails with 'str' object has no attribute 'get' when called via Anthropic endpoint

[Moep90]

What happened?

When routing Anthropic-style requests from Claude Code to an Ollama model through LiteLLM, completions fail with a 500 due to a AttributeError: 'str' object has no attribute 'get'. This appears to be caused by LiteLLM’s Ollama prompt-template handler expecting messages[*].content to be a list of content blocks instead of a string.

Additional context:
I also tried with several other models:

1. qwen3: 8b
2. qwen3: 0.6b
3. qwen2.5-coder: 1.5b
4. qwen2.5-coder: 3b
5. qwen2.5-coder: 7b
6. llama3-groq-tool-use: 8b
7. hhao/qwen2.5-coder-tools: 7b
8. granite3-dense: 2b

Workstation config:

$  env | grep ANTHROPIC
ANTHROPIC_BASE_URL=http://<litellm url>:4000
ANTHROPIC_API_KEY=your-expected-anthropic-api-key
ANTHROPIC_AUTH_TOKEN=sk-1234567890

$ cat litellm/config.yaml

---
model_list:
  - model_name: "*"
    litellm_params:
      model: ollama/qwen2.5-coder:7b
      keep_alive: "8m"
      api_base: http://<ollama url>:<ollama port>
      model_info:
        supports_function_calling: true

litellm_settings:
  master_key: "sk-1234567890"

Relevant log output

Workstation Claude log:


> /init is analyzing your codebase…
  ⎿  API Error (500 {"error":{"message":"Error calling litellm.acompletion for non-Anthropic model: litellm.APIConnectionError: 'str' object has no attribute 'get'\nTraceback (most recent call last):\n  File \"/usr/lib/python3.13/site-packages/litellm/main.py\", line 3086, in completion\n    response = base_llm_http_handler.completion(\n        model=model,\n    ...<13 lines>...\n        client=client,\n    )\n  File \"/usr/lib/python3.13/site-packages/litellm/llms/custom_httpx/llm_http_handler.py\", line 329, in completion\n    data = provider_config.transform_request(\n        model=model,\n    ...<3 lines>...\n        headers=headers,\n    )\n  File \"/usr/lib/python3.13/site-packages/litellm/llms/ollama/completion/transformation.py\", line 342, in transform_request\n    modified_prompt = ollama_pt(model=model, messages=messages)\n  File \"/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py\", line 222, in ollama_pt\n    system_content_str, msg_i = _handle_ollama_system_message(\n                                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^\n        messages, prompt, msg_i\n        ^^^^^^^^^^^^^^^^^^^^^^^\n    )\n    ^\n  File \"/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py\", line 180, in _handle_ollama_system_message\n    msg_content = convert_content_list_to_str(messages[msg_i])\n  File \"/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/common_utils.py\", line 132, in convert_content_list_to_str\n    text_content = c.get(\"text\")\n                   ^^^^^\nAttributeError: 'str' object has no attribute 'get'\n","type":"None","param":"None","code":"500"}}) · Retrying in 1 seconds… (attempt 1/10)


LiteLLM log:


litellm-1    | 16:01:16 - LiteLLM Router:INFO: router.py:2779 - ageneric_api_call_with_fallbacks(model=hhao/qwen2.5-coder-tools:7b) Exception Error calling litellm.acompletion for non-Anthropic model: litellm.APIConnectionError: 'str' object has no attribute 'get'
litellm-1    | Traceback (most recent call last):
litellm-1    |   File "/usr/lib/python3.13/site-packages/litellm/main.py", line 3086, in completion
litellm-1    |     response = base_llm_http_handler.completion(
litellm-1    |         model=model,
litellm-1    |     ...<13 lines>...
litellm-1    |         client=client,
litellm-1    |     )
litellm-1    |   File "/usr/lib/python3.13/site-packages/litellm/llms/custom_httpx/llm_http_handler.py", line 329, in completion
litellm-1    |     data = provider_config.transform_request(
litellm-1    |         model=model,
litellm-1    |     ...<3 lines>...
litellm-1    |         headers=headers,
litellm-1    |     )
litellm-1    |   File "/usr/lib/python3.13/site-packages/litellm/llms/ollama/completion/transformation.py", line 342, in transform_request
litellm-1    |     modified_prompt = ollama_pt(model=model, messages=messages)
litellm-1    |   File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 222, in ollama_pt
litellm-1    |     system_content_str, msg_i = _handle_ollama_system_message(
litellm-1    |                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
litellm-1    |         messages, prompt, msg_i
litellm-1    |         ^^^^^^^^^^^^^^^^^^^^^^^
litellm-1    |     )
litellm-1    |     ^
litellm-1    |   File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/factory.py", line 180, in _handle_ollama_system_message
litellm-1    |     msg_content = convert_content_list_to_str(messages[msg_i])
litellm-1    |   File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/prompt_templates/common_utils.py", line 132, in convert_content_list_to_str
litellm-1    |     text_content = c.get("text")
litellm-1    |                    ^^^^^
litellm-1    | AttributeError: 'str' object has no attribute 'get'
litellm-1    |

Are you a ML Ops Team?

Yes

What LiteLLM version are you on ?

1.74.15

Twitter / LinkedIn details

No response

[superpoussin22]

please add a title :)

[Moep90]

You've been too quick :-), done.

[tanish1729]

hi! i'd like to take on this bug and raise a fix for it. could you help me out in getting started?

[Moep90]

Hey guys,

Is there any chance to get this resolved soon(ish)?

[NiklasRosenstein]

I did some digging as to what's going wrong here.

This condition evaluates to true when using the Anthropic API with tools and routing to an Ollama endpoint, independent of whether supports_function_calling is enabled or not. I don't know if this is supposed to happen with supports_function_calling enabled, but that's what happens.

litellm/litellm/main.py

Lines 1259 to 1267 in 5299e9a

	if litellm.add_function_to_prompt and optional_params.get(
	"functions_unsupported_model", None
	): # if user opts to add it to prompt, when API doesn't support function calling
	functions_unsupported_model = optional_params.pop(
	"functions_unsupported_model"
	)
	messages = function_call_prompt(
	messages=messages, functions=functions_unsupported_model
	)

(side note: @Moep90 Your model_info in your config is indented one tab-width too many; it goes next to litellm_params, not under it).

The function generates an additional message to inform the model of the available tools.

litellm/litellm/litellm_core_utils/prompt_templates/factory.py

Lines 3804 to 3818 in 5299e9a

	def function_call_prompt(messages: list, functions: list):
	function_prompt = """Produce JSON OUTPUT ONLY! Adhere to this format {"name": "function_name", "arguments":{"argument_name": "argument_value"}} The following functions are available to you:"""
	for function in functions:
	function_prompt += f"""\n{function}\n"""
	function_added_to_prompt = False
	for message in messages:
	if "system" in message["role"]:
	message["content"] += f""" {function_prompt}"""
	function_added_to_prompt = True
	if function_added_to_prompt is False:
	messages.append({"role": "system", "content": f"""{function_prompt}"""})
	return messages

But it handles the message incorrectly. If we dump the entire messages that is sent by Claude Code (with some redaction to keep it smaller), we get:

[
  {
    "role": "system",
    "content": [
      {
        "type": "text",
        "text": "You are Claude Code, Anthropic's official CLI for Claude.",
        "cache_control": {
          "type": "ephemeral"
        }
      },
      {
        "type": "text",
        "text": "\nYou are an interactive CLI tool that helps users with software...",
        "cache_control": {
          "type": "ephemeral"
        }
      }
    ]
  },
  {
    "role": "user",
    "content": [
      {
        "type": "text",
        "text": "<system-reminder>\nAs you answer the user's questions, you can use the ..."
      },
      {
        "type": "text",
        "text": "Test"
      },
      {
        "type": "text",
        "text": "<system-reminder>\nThis is a reminder that your todo list ..."
      }
    ]
  }
]

https://github.com/BerriAI/litellm/blob/5299e9ad4ae9acae59decb5eb995693ec7920c2b/litellm/litellm_core_utils/prompt_templates/factory.py#L3812\

So the message["content"] here is not a string, but instead

[
  {
    "type": "text",
    "text": "You are Claude Code, Anthropic's official CLI for Claude.",
    "cache_control": {
      "type": "ephemeral"
    }
  },
  {
    "type": "text",
    "text": "\nYou are an interactive CLI tool that helps users with software...",
    "cache_control": {
      "type": "ephemeral"
    }
  }
]

I can't tell whether it is valid at this point to be receiving the messages in this format and the function needs to update how it handles the messages, or whether the messages must be transformed before they are passed into the function. Considering however that it seems to work fine when sending requests without tools, likely the function needs to be updated to handle the message differently if it is a list.

[NiklasRosenstein]

This injected prompt also doesn't seem particularly useful to me 🤔 That's because it's instructing the model to only respond in JSON, and only in a format compatible with the tools. Assuming the model closely follows those instructions, it could never respond with anything but a tool call.

The Python anthropic SDK seems to understand that the plain JSON response is a tool call, but Claude Code doesn't seem to recognize it as such anyway (I just get a plain dump of the tool call as output and it stops there).

A lot of models used with Ollama do support tools natively however, so there's definitely something funky also with the detection of whether to inject this prompt or not.

[NiklasRosenstein]

I got tool calling working using the ollama_chat provider when testing with the Python anthropic SDK. The documentation also mentions this provider as the suggested one to use instead of ollama "for better responses".

Note that setting api_base in the litellm_params is not sufficient and the OLLAMA_API_BASE environment variable must also be set. While the parameter seems not completely ignored, the code that looks up the environment variable here does not appear to try and use the api_key parameter first (and it doesn't look like it has easy access to that parameter).

litellm/litellm/llms/ollama/completion/transformation.py

Lines 204 to 212 in 55dcade

	def get_model_info(self, model: str) -> ModelInfoBase:
	"""
	curl http://localhost:11434/api/show -d '{
	"name": "mistral"
	}'
	"""
	if model.startswith("ollama/") or model.startswith("ollama_chat/"):
	model = model.split("/", 1)[1]
	api_base = get_secret_str("OLLAMA_API_BASE") or "http://localhost:11434"

# litellm-config.yaml
  - model_name: qwen3-coder
    litellm_params:
      model: ollama_chat/qwen3-coder
      api_base: http://localhost:11434
    model_info:
      supports_function_calling: true

And then run

$ OLLAMA_API_BASE=http://localhost:11434 uvx litellm[proxy] --config litellm-config.yaml

Anthropic SDK test script

from anthropic import Anthropic

client = Anthropic(
    base_url="http://localhost:4000",
    api_key="ignored",
)

for message in client.messages.create(
    max_tokens=1024,
    stream=False,
    messages=[
        {
            "role": "user",
            "content": "How are you? Greet John with his name and age 30.",
        }
    ],
    tools=[
        {
            "input_schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "age": {"type": "integer"},
                },
                "required": ["name", "age"],
            },
            "name": "greet",
            "description": "A tool to greet someone with their name and age.",
            "cache_control": {"ephemeral": True},
            "type": "custom",
        }
    ],
    model="qwen3-coder",
):
    print(message)

It seems that ollama_chat runs into another issue when used with Claude Code however:

  File "/home/niklas/tmp/llmproxy/litellm/litellm/llms/anthropic/experimental_pass_through/adapters/handler.py", line 179, in async_anthropic_messages_handler
    raise ValueError(
ValueError: Error calling litellm.acompletion for non-Anthropic model: litellm.APIConnectionError: Ollama_chatException - {"error":"json: cannot unmarshal array into Go struct field ChatRequest.messages.content of type string"}

[NiklasRosenstein]

The two errors are related; both should be fixed here: #13833 Will update the PR description another time.