fix(bedrock): correct max tokens for meta.llama3-70b-instruct-v1:0

[mngue]

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

This is a static data fix to model_prices_and_context_window.json. meta.llama3-70b-instruct-v1:0 is listed in litellm's BEDROCK_CONVERSE_MODELS, so bedrock/meta.llama3-70b-instruct-v1:0 (no explicit route prefix) always routes through the Bedrock Converse API by default.

A real request against a live us-east-1 endpoint with max_tokens above 2048, routed through Converse, returns:

litellm.BadRequestError: BedrockException - {"message":"The maximum tokens you requested exceeds the model limit of 2048. Try again with a maximum tokens value that is lower than 2048."}

Before this fix, most regional entries (ap-south-1, ca-central-1, eu-west-1, eu-west-2, sa-east-1, us-east-1, us-west-1) and the global meta.llama3-70b-instruct-v1:0 entry listed max_output_tokens: 8192 / max_tokens: 8192, which is 4x the actual 2048 output cap enforced by the Converse route.

Separately, the two GovCloud entries (us-gov-east-1, us-gov-west-1) already had the correct max_output_tokens: 2048, but listed max_input_tokens: 8000 instead of 8192. Requesting a prompt near the boundary against a live us-east-1 endpoint confirms the context length is 8192, not 8000:

litellm.llms.bedrock.common_utils.BedrockError: This model's maximum context length is 8192 tokens. Please reduce the length of the prompt

Before (on main, example for us-east-1):

"bedrock/us-east-1/meta.llama3-70b-instruct-v1:0": {
    "max_input_tokens": 8192,
    "max_output_tokens": 8192,
    "max_tokens": 8192,
    ...
}

After (this PR):

"bedrock/us-east-1/meta.llama3-70b-instruct-v1:0": {
    "max_input_tokens": 8192,
    "max_output_tokens": 2048,
    "max_tokens": 2048,
    ...
}

Note: the underlying Llama 3 70B model may accept a higher max_gen_len (tested up to 4096) through Bedrock's native Invoke API. However, since this model is hardcoded to the Converse route in litellm and litellm's Llama Invoke transformation currently throws an unrelated ValidationException for this model, 2048 reflects the actual limit for the only reachable, working path today. The Invoke-route issue is a separate bug outside this PR's scope.

Verified both model_prices_and_context_window.json and the synced litellm/model_prices_and_context_window_backup.json remain valid JSON after the change:

python3 -c "import json; json.load(open('model_prices_and_context_window.json'))"
python3 -c "import json; json.load(open('litellm/model_prices_and_context_window_backup.json'))"

Type

🐛 Bug Fix

Changes

• Fixed max_output_tokens/max_tokens for meta.llama3-70b-instruct-v1:0 from 8192 to 2048 across all 7 non-GovCloud regional entries and the global entry, matching the Converse route's enforced output cap.
• Fixed max_input_tokens for the two GovCloud entries (us-gov-east-1, us-gov-west-1) from 8000 to 8192, matching the confirmed context length and the other regions.
• Applied the same fixes to litellm/model_prices_and_context_window_backup.json to keep it in sync with model_prices_and_context_window.json.

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

This PR fixes incorrect token limit metadata for meta.llama3-70b-instruct-v1:0 on Bedrock, where the Converse route enforces a 2048-token output cap that was previously misrepresented as 8192 across most entries.

• Output tokens corrected (8192 → 2048): All 7 non-GovCloud regional entries (ap-south-1, ca-central-1, eu-west-1, eu-west-2, sa-east-1, us-east-1, us-west-1) and the global meta.llama3-70b-instruct-v1:0 entry have max_output_tokens and max_tokens set to 2048, matching the live Converse API limit.
• Input tokens corrected (8000 → 8192): The two GovCloud entries (us-gov-east-1, us-gov-west-1) had a separate typo in max_input_tokens; the correct context length is 8192, confirmed against the live endpoint error.
• Both model_prices_and_context_window.json and litellm/model_prices_and_context_window_backup.json are updated identically, keeping them in sync.

Confidence Score: 5/5

Safe to merge — data-only corrections to two JSON files with no logic changes and no test regressions.

Every affected entry for meta.llama3-70b-instruct-v1:0 has been identified and corrected, both JSON files are kept in sync, the author provided live-endpoint error messages confirming both the 2048 output cap and the 8192 input limit, and no existing tests assert the old (incorrect) values.

No files require special attention.

Important Files Changed

Filename	Overview
model_prices_and_context_window.json	Corrects max_output_tokens/max_tokens from 8192 to 2048 for all 7 non-GovCloud regional entries and the global meta.llama3-70b-instruct-v1:0 entry; corrects max_input_tokens from 8000 to 8192 for the two GovCloud entries. All 10 model entries for this model are accounted for and updated correctly.
litellm/model_prices_and_context_window_backup.json	Identical token-limit corrections applied to keep the backup in sync with model_prices_and_context_window.json; changes are byte-for-byte consistent between the two files.

_{Reviews (1): Last reviewed commit: "fix(bedrock): correct max_input_tokens/m..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!