fix(anthropic): honor messages request timeout

[yucheng-berri]

Relevant issues

Fixes #26752

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Proof below was captured by the original author on the original PR's commits. The upstream is a local Anthropic-compatible server that sleeps for 1.25s while the model config sets request_timeout: 0.3, which keeps the proof deterministic

Config used for both runs:

model_list:
  - model_name: claude-timeout-proof
    litellm_params:
      model: anthropic/claude-timeout-proof
      api_key: sk-ant-local-proof
      api_base: http://127.0.0.1:8999
      request_timeout: 0.3

general_settings:
  master_key: sk-1234

litellm_settings:
  drop_params: true
  modify_params: true
  num_retries: 0

Before fix, at base commit bf02a4a47f, the request waited for the slow upstream and returned 200, showing that the model-level timeout was not applied to the Anthropic /v1/messages HTTP request:

uv run --extra proxy python litellm/proxy/proxy_cli.py --config /private/tmp/litellm_timeout_proof_config.yaml --port 4010 --detailed_debug

curl -sS -w '\nhttp_code=%{http_code}\ntime_total=%{time_total}\n' \
  -H 'Authorization: Bearer sk-1234' \
  -H 'Content-Type: application/json' \
  -H 'anthropic-version: 2023-06-01' \
  --data '{"model":"claude-timeout-proof","max_tokens":1,"messages":[{"role":"user","content":"hi"}]}' \
  http://127.0.0.1:4010/v1/messages

{"id":"msg_timeout_proof","type":"message","role":"assistant","model":"claude-timeout-proof","content":[{"type":"text","text":"slow upstream response"}],"stop_reason":"end_turn","stop_sequence":null,"usage":{"input_tokens":5,"output_tokens":3}}
http_code=200
time_total=1.946157

After fix, at original commit 63019add86, the same request returns a LiteLLM timeout. The response body shows the configured 0.3s timeout reached the HTTP request path:

{"error":{"message":"litellm.Timeout: Connection timed out. Timeout passed=0.3, time taken=0.302 seconds. Received Model Group=claude-timeout-proof\nAvailable Model Group Fallbacks=None","type":"None","param":"None","code":"408"}}
http_code=408
time_total=1.512531

Type

Bug Fix

Changes

Anthropic /v1/messages now resolves per-request and configured timeouts through the shared completion timeout resolver, then forwards the resolved value into the async HTTP POST helper, on the first attempt and on every retry

On top of the original change, the follow-up commit makes the resolver return None when no timeout was configured anywhere (no model-level timeout or stream_timeout, no deployment request_timeout, no explicitly set litellm.request_timeout). The POST then falls back to the cached client's default httpx.Timeout(600, connect=5) instead of an explicit bare 600.0, which would have set the connect timeout to 600s as well

Regression coverage checks timeout precedence (model timeout beats deployment request_timeout, stream_timeout only applies to streaming requests), the unconfigured fallback returning None, and the full async /v1/messages handler wiring into the HTTP client for both streaming and non-streaming requests. Each new assertion was mutation-checked: always reading stream_timeout, hard-coding stream=False at the call site, and dropping the None fallback each fail exactly one of the new tests

Behavior changes

Requests to /v1/messages that set a timeout (request body, deployment config, or an explicit litellm.request_timeout) now honor it; previously it was ignored and the client default of 600s applied. When the resolved timeout is a bare number, httpx applies it to connect/read/write/pool alike, matching the existing chat completions behavior. When no timeout is configured anywhere, behavior is unchanged: the client default of 600s read with a 5s connect timeout still applies

Credit

Adopted from #32827 by @MelvinOrichiSocana-hs. Mirrored onto a litellm_ branch so CircleCI and the internal lint workflow run

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Low Risk
Scoped to Anthropic messages HTTP wiring; unconfigured requests keep existing client defaults, while configured timeouts may surface 408s where slow calls previously succeeded.

Overview
Anthropic /v1/messages async HTTP calls now apply the same timeout resolution as chat completions instead of always using the client’s long default.

A new _resolve_anthropic_messages_timeout helper picks stream_timeout on streaming requests, otherwise timeout, then deployment request_timeout, then an explicit global litellm.request_timeout, via CompletionTimeout.resolve. That value is passed into _async_post_anthropic_messages_with_http_error_retry on the initial POST and retries. If nothing is configured at any layer, the resolver returns None so httpx keeps the cached client default (600s read / 5s connect) rather than forcing a bare numeric timeout that would widen connect.

Tests cover precedence, the unconfigured None path, and end-to-end forwarding for non-streaming and streaming handlers.

^{Reviewed by Cursor Bugbot for commit c3eb9dc. Bugbot is set up for automated code reviews on this repo. Configure here.}

[yucheng-berri]

@greptileai

[greptile-apps]

Greptile Summary

This PR fixes the Anthropic /v1/messages path ignoring per-request and deployment-level timeouts, causing every request to silently use the httpx client's 600-second default. The fix adds _resolve_anthropic_messages_timeout, which mirrors the shared CompletionTimeout.resolve logic already used by chat completions, and passes the result into the HTTP retry helper.

• _resolve_anthropic_messages_timeout picks stream_timeout for streaming requests, otherwise timeout, then deployment request_timeout, then an explicit global litellm.request_timeout, and returns None when nothing is configured — preserving the client-level httpx.Timeout(600, connect=5) default rather than replacing it with a bare 600-second float.
• The resolved value is forwarded on both the first attempt and every retry inside _async_post_anthropic_messages_with_http_error_retry, and the existing FakeAsyncClient in one test gets the new timeout=None kwarg in its signature to stay compatible.

Confidence Score: 5/5

Safe to merge; the change only affects requests that already had a configured timeout — those that had none continue to use the same 600s/5s-connect client default as before.

The timeout resolution reuses CompletionTimeout.resolve, which is already trusted by the chat-completions path. The None guard correctly short-circuits before calling that resolver, leaving unconfigured requests untouched. AsyncHTTPHandler.post already handles timeout=None by falling back to self.timeout, so no edge case is introduced there. The three new parametrized tests were mutation-checked by the author and cover the full precedence ladder and the unconfigured fallback.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/llms/custom_httpx/llm_http_handler.py	Adds _resolve_anthropic_messages_timeout static method and threads the resolved timeout through _async_post_anthropic_messages_with_http_error_retry on first attempt and all retries; None is correctly returned when no timeout is configured anywhere, so the client keeps its httpx.Timeout(600, connect=5) default.
tests/test_litellm/llms/custom_httpx/test_llm_http_handler.py	Adds three new tests covering timeout precedence (test_resolve_anthropic_messages_timeout), request_timeout forwarding, and stream_timeout forwarding; updates FakeAsyncClient.post signature to accept the new timeout kwarg without breaking the existing Bedrock signing retry test.

_{Reviews (4): Last reviewed commit: "Merge remote-tracking branch 'origin/lit..." | Re-trigger Greptile}

[greptile-apps]

Greptile Summary

This PR fixes the Anthropic /v1/messages pass-through handler so that per-model and per-deployment timeouts (timeout, stream_timeout, request_timeout, litellm.request_timeout) are correctly resolved and forwarded to the underlying httpx POST, matching the existing behavior of the completion() path. When no timeout is configured anywhere, None is passed so the cached client's httpx.Timeout(600, connect=5) default is used instead of a bare 600.0 (which would have applied a 600-second connect timeout).

• Adds _resolve_anthropic_messages_timeout static method to BaseLLMHTTPHandler that delegates to the shared CompletionTimeout.resolve / get_configured_request_timeout infrastructure, and pipes the result through to every HTTP POST attempt including retries.
• Adds a parametrized unit test for the resolver covering all precedence rules, plus two async integration tests verifying request_timeout and stream_timeout reach the httpx client; also updates a pre-existing FakeAsyncClient.post stub to accept the now-required timeout keyword argument.

Confidence Score: 5/5

Safe to merge — the change is narrowly scoped to the Anthropic messages pass-through path, reuses the shared timeout resolution infrastructure, and does not alter behavior when no timeout is configured.

The resolver correctly short-circuits to None when no timeout is set anywhere, so the cached client httpx.Timeout(600, connect=5) default is preserved. Timeout precedence mirrors the existing completion() path. The FakeAsyncClient.post update adds timeout=None only to satisfy the new keyword argument — no assertion is removed or weakened.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/llms/custom_httpx/llm_http_handler.py	Adds timeout parameter to _async_post_anthropic_messages_with_http_error_retry and a new _resolve_anthropic_messages_timeout static method that correctly resolves per-request/model/global timeouts, then wires the resolved value into the Anthropic messages HTTP call path.
tests/test_litellm/llms/custom_httpx/test_llm_http_handler.py	Adds parametrized unit tests for _resolve_anthropic_messages_timeout covering timeout precedence and the None fallback, two async integration tests confirming request_timeout and stream_timeout reach the HTTP POST call, and updates the existing FakeAsyncClient.post stub to accept the new timeout keyword argument.

_{Reviews (2): Last reviewed commit: "fix(anthropic): keep client default conn..." | Re-trigger Greptile}

[yucheng-berri]

@greptileai

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[yucheng-berri]

bugbot run

[yucheng-berri]

The failing documentation check is pre-existing on the base branch: tests/documentation_tests/test_api_docs.py requires every UpdateTeamRequest field to be documented in the update_team docstring, and mcp_rpm_limit is missing from that docstring on litellm_internal_staging itself (litellm/proxy/management_endpoints/team_endpoints.py:1582; the field is documented for new_team at line 928 but was never added to update_team). This PR does not touch that file. A separate one-line docstring fix is being opened against litellm_internal_staging; once it merges this branch will be updated and the check re-run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit d8e31ce. Configure here.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_anthropic_messages_timeout (c3eb9dc) with litellm_internal_staging (9c59e2a)}

[yucheng-berri]

@greptileai

[yucheng-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit c3eb9dc. Configure here.}