Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR adds Bedrock Devstral 2 (123B) model definitions across three files. It registers
Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| litellm/constants.py | Adds mistral.devstral-2-123b to the BEDROCK_CONVERSE_MODELS list. Correctly placed among other Mistral models. No issues found. |
| model_prices_and_context_window.json | Adds pricing/config entries for mistral.devstral-2-123b across 12 AWS regions plus a base entry. Entries include flex/priority pricing tiers, consistent structure, and correct bedrock_converse provider. Missing supports_system_messages flag that other Mistral models include. |
| litellm/model_prices_and_context_window_backup.json | Identical changes to model_prices_and_context_window.json (backup copy). Both files are in sync. Same minor observation about missing supports_system_messages. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["User calls litellm.completion(model='bedrock/us-east-1/mistral.devstral-2-123b')"] --> B["get_llm_provider_logic.py\nResolves provider as 'bedrock'"]
B --> C["bedrock/common_utils.py\nChecks BEDROCK_CONVERSE_MODELS"]
C --> D{"Is model in\nbedrock_converse_models?"}
D -->|Yes| E["Route to Bedrock Converse API"]
D -->|No| F["Route to Bedrock Invoke API"]
E --> G["model_prices_and_context_window.json\nLookup pricing & capabilities"]
G --> H["Return response with\ncost tracking"]
Last reviewed commit: fa102a9
| "supports_assistant_prefill": true, | ||
| "supports_function_calling": true, | ||
| "supports_response_schema": true, | ||
| "supports_tool_choice": true |
There was a problem hiding this comment.
Missing supports_system_messages flag
Other Mistral bedrock_converse models (e.g., mistral.magistral-small-2509, mistral.ministral-3-14b-instruct) include "supports_system_messages": true. This entry omits it. While it defaults to None and won't cause a runtime error, adding it would maintain consistency and ensure system messages are correctly handled for this model.
| "supports_assistant_prefill": true, | |
| "supports_function_calling": true, | |
| "supports_response_schema": true, | |
| "supports_tool_choice": true | |
| "supports_assistant_prefill": true, | |
| "supports_function_calling": true, | |
| "supports_response_schema": true, | |
| "supports_system_messages": true, | |
| "supports_tool_choice": true |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
Pull request overview
Adds pricing/context-window metadata for the Bedrock-hosted Mistral Devstral 2 123B model so LiteLLM can compute costs and route requests correctly (via bedrock_converse).
Changes:
- Added regional Bedrock pricing entries for
mistral.devstral-2-123bacross multiple AWS regions. - Added a base (non-region) model entry
mistral.devstral-2-123bfor default lookups. - Registered
mistral.devstral-2-123binBEDROCK_CONVERSE_MODELSso it’s recognized as a Converse-routed Bedrock model.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| model_prices_and_context_window.json | Adds Devstral 2 123B pricing + limits + capability flags (regional + base entry). |
| litellm/model_prices_and_context_window_backup.json | Keeps the backup JSON in sync with the main pricing file. |
| litellm/constants.py | Adds the model ID to BEDROCK_CONVERSE_MODELS for Bedrock Converse routing/recognition. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Adds Bedrock Devstral 2 DB entries
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewCI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🆕 New Feature
Changes