Open
Conversation
… deployment tagging (#21655) POST /access_group/new and PUT /access_group/{name}/update now accept an optional model_ids list that targets specific deployments by their unique model_id, instead of tagging every deployment that shares a model_name. When model_ids is provided it takes priority over model_names, giving API callers the same single-deployment precision that the UI already has via PATCH /model/{model_id}/update. Backward compatible: model_names continues to work as before. Closes #21544
… custom favicon for the litellm proxy UI.\n\n- Add favicon_url field to UIThemeConfig model\n- Add LITELLM_FAVICON_URL env var support\n- Add /get_favicon endpoint to serve custom favicons\n- Update ThemeContext to dynamically set favicon\n- Add favicon URL input to UI theme settings page\n- Add comprehensive tests\n\nCloses #8323 (#21653)
In create_file for Bedrock, get_complete_file_url is called twice: once in the sync handler (generating UUID-1 for api_base) and once inside transform_create_file_request (generating UUID-2 for the actual S3 upload). The Bedrock provider correctly writes UUID-2 into litellm_params["upload_url"], but the sync handler unconditionally overwrites it with api_base (UUID-1). This causes the returned file_id to point to a non-existent S3 key. Fix: only set upload_url to api_base when transform_create_file_request has not already set it, preserving the Bedrock provider's value. Closes #21546
…nt (#21649) Add vector_size parameter to QdrantSemanticCache and expose it through the Cache facade as qdrant_semantic_cache_vector_size. This allows users to use embedding models with dimensions other than the default 1536, enabling cheaper/stronger models like Stella (1024d), bge-en-icl (4096d), voyage, cohere, etc. The parameter defaults to QDRANT_VECTOR_SIZE (env var or 1536) for backward compatibility. When creating new collections, the configured vector_size is used instead of the hardcoded constant. Closes #9377
…1762) Clients like OpenCode's @ai-sdk/openai-compatible send budgetTokens (camelCase) instead of budget_tokens in the thinking parameter, causing validation errors. Add early normalization in completion().
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
Contributor
Greptile SummaryStaging PR bundling six independent changes: (1) custom favicon support for the admin UI, (2) configurable vector dimensions for Qdrant semantic cache, (3) Bedrock batch file transformation fix to prevent double UUID in S3 keys and add Nova/Converse API support, (4)
Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| litellm/proxy/proxy_server.py | New /get_favicon endpoint for custom favicon support. Follows the same pattern as get_image but lacks response caching for remote URLs. |
| litellm/llms/bedrock/files/transformation.py | Adds Converse API (Nova) provider support for batch inference transformation, with correct routing between Anthropic, Converse, and passthrough paths. Well-structured refactoring with a clear class-level constant for provider dispatch. |
| litellm/llms/custom_httpx/llm_http_handler.py | Fixes sync create_file to honour the upload_url already set by transform_create_file_request (e.g. Bedrock S3 pre-signed URLs), instead of always overwriting it with api_base. |
| litellm/caching/qdrant_semantic_cache.py | Adds configurable vector_size parameter, defaulting to QDRANT_VECTOR_SIZE constant. Clean change that stores the value as an instance attribute and uses it in collection creation. |
| litellm/utils.py | New validate_and_fix_thinking_param function that normalizes camelCase budgetTokens to snake_case budget_tokens. Does not mutate the original dict. Handles edge cases of both keys being present. |
| litellm/main.py | Integrates validate_and_fix_thinking_param into the completion() flow to normalize camelCase thinking keys before processing. |
| litellm/proxy/management_endpoints/model_access_group_management_endpoints.py | Adds model_ids support for precise deployment tagging in access group endpoints. Both create_model_group and update_access_group now accept either model_names or model_ids, with model_ids taking priority when both are provided. |
| litellm/types/proxy/management_endpoints/model_management_endpoints.py | Adds optional model_ids field to NewModelGroupRequest, NewModelGroupResponse, and UpdateModelGroupRequest. Makes model_names optional (was required) to support the new model_ids path. |
| ui/litellm-dashboard/src/components/ui_theme_settings.tsx | Adds favicon URL input to UI theme settings. Removes logo preview section. Sends favicon_url alongside logo_url in save/reset operations. |
| ui/litellm-dashboard/src/contexts/ThemeContext.tsx | Extends ThemeContext with faviconUrl state. Adds a useEffect that dynamically updates the document's favicon link elements when faviconUrl changes. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
subgraph Favicon["Custom Favicon Flow"]
A["/get_favicon endpoint"] --> B{LITELLM_FAVICON_URL set?}
B -->|No| C{Default favicon exists?}
C -->|Yes| D[Serve default favicon.ico]
C -->|No| E[404 Not Found]
B -->|Yes - HTTP URL| F[Fetch remote URL]
F -->|Success| G[Return Response with content-type]
F -->|Failure| C
B -->|Yes - Local path| H{File exists?}
H -->|Yes| I[Serve local file]
H -->|No| C
end
subgraph Bedrock["Bedrock Batch Transformation"]
J["_map_openai_to_bedrock_params"] --> K{Provider?}
K -->|Anthropic| L["AmazonAnthropicClaudeConfig.transform_request"]
K -->|Nova| M["AmazonConverseConfig.transform_request"]
K -->|Other| N["Passthrough: messages + params"]
end
subgraph AccessGroup["Access Group model_ids"]
O["create_model_group / update_access_group"] --> P{model_ids provided?}
P -->|Yes| Q["update_specific_deployments_with_access_group\n(tags only specified model_ids)"]
P -->|No| R["update_deployments_with_access_group\n(tags ALL deployments by model_name)"]
end
Last reviewed commit: cc6ef0e
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+10764
to
+10783
| if favicon_url.startswith(("http://", "https://")): | ||
| try: | ||
| from litellm.llms.custom_httpx.http_handler import ( | ||
| get_async_httpx_client, | ||
| ) | ||
| from litellm.types.llms.custom_http import httpxSpecialProvider | ||
|
|
||
| async_client = get_async_httpx_client( | ||
| llm_provider=httpxSpecialProvider.UI, | ||
| params={"timeout": 5.0}, | ||
| ) | ||
| response = await async_client.get(favicon_url) | ||
| if response.status_code == 200: | ||
| content_type = response.headers.get( | ||
| "content-type", "image/x-icon" | ||
| ) | ||
| return Response( | ||
| content=response.content, | ||
| media_type=content_type, | ||
| ) |
Contributor
There was a problem hiding this comment.
No caching for remote favicon fetches
Unlike the get_image endpoint (line 10711-10732) which caches downloaded logos to a local file, this get_favicon endpoint re-fetches the remote URL on every request. Since browsers request favicons frequently, this will generate an outbound HTTP call for every page load when a remote LITELLM_FAVICON_URL is configured. Consider adding caching similar to get_image.
Adds per-alert-type digest mode that aggregates duplicate alerts
within a configurable time window and emits a single summary message
with count, start/end timestamps.
Configuration via general_settings.alert_type_config:
alert_type_config:
llm_requests_hanging:
digest: true
digest_interval: 86400
Digest key: (alert_type, request_model, api_base)
Default interval: 24 hours
Window type: fixed interval
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewCI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes