feat: migrate Bedrock Claude models to native Anthropic Messages API#1924
Open
Pratham-Mishra04 wants to merge 1 commit intomainfrom
Open
Conversation
18 tasks
Contributor
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe PR routes Anthropic-compatible models in the Bedrock provider to Anthropic-style InvokeModel endpoints while keeping other models on Bedrock's Converse API. It adds Bedrock-specific Anthropic request marshaling helpers and exposes two Anthropic responses stream pool accessors; changelogs and tests updated accordingly. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant BedrockProvider as Bedrock Provider
participant DeploymentResolver as Deployment Resolver
participant AnthropicPath as Anthropic Path
participant ConverseAPI as Converse API
participant ResponseParser
Client->>BedrockProvider: ChatCompletion Request
BedrockProvider->>DeploymentResolver: Resolve deployment for model
DeploymentResolver-->>BedrockProvider: Anthropic? / Converse?
alt Anthropic
BedrockProvider->>AnthropicPath: Build Anthropic InvokeModel body
AnthropicPath->>BedrockProvider: Marshaled Request
BedrockProvider->>BedrockProvider: Call InvokeModel (invoke)
BedrockProvider->>ResponseParser: Parse Anthropic response
ResponseParser-->>BedrockProvider: Enriched Bifrost response
else Converse
BedrockProvider->>ConverseAPI: Build Converse request
ConverseAPI->>BedrockProvider: Request ready
BedrockProvider->>BedrockProvider: Call Converse endpoint
BedrockProvider->>ResponseParser: Parse Bedrock response
ResponseParser-->>BedrockProvider: Enriched Bifrost response
end
BedrockProvider-->>Client: Bifrost ChatCompletion Response
sequenceDiagram
participant Client
participant BedrockProvider as Bedrock Provider
participant DeploymentResolver as Deployment Resolver
participant AnthropicStream as Anthropic Stream
participant ConverseStream as Converse Stream
participant ChunkProcessor as Chunk Processor
Client->>BedrockProvider: ChatCompletionStream Request
BedrockProvider->>DeploymentResolver: Resolve deployment
DeploymentResolver-->>BedrockProvider: Anthropic? / Converse?
alt Anthropic
BedrockProvider->>AnthropicStream: Invoke invoke-with-response-stream
loop For each event
AnthropicStream->>ChunkProcessor: Deliver stream event
ChunkProcessor->>ChunkProcessor: Parse & accumulate usage
ChunkProcessor-->>BedrockProvider: Enriched chunk
BedrockProvider-->>Client: Stream chunk
end
else Converse
BedrockProvider->>ConverseStream: Invoke converse-stream
loop For each event
ConverseStream->>ChunkProcessor: Deliver stream event
ChunkProcessor->>ChunkProcessor: Parse & accumulate usage
ChunkProcessor-->>BedrockProvider: Enriched chunk
BedrockProvider-->>Client: Stream chunk
end
end
ChunkProcessor-->>Client: Finalization with metadata
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
Comment |
Collaborator
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
18 tasks
akshaydeo
reviewed
Mar 5, 2026
| ImageEdit: true, | ||
| ImageVariation: true, | ||
| StructuredOutputs: true, | ||
| // BatchCreate: true, |
Collaborator
Author
There was a problem hiding this comment.
no, it was supposed to be a draft PR mb
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (2)
core/providers/bedrock/anthropic_compat.go (1)
83-85: Avoid mutating the incoming request object in-place.This helper changes
request.Model, which can leak side effects to callers that reuse/log the original request. Use a shallow copy before overriding model.🛠️ Proposed fix
- // Mutate the model before conversion so converters see the resolved deployment name - request.Model = deployment - reqBody, err := anthropic.ToAnthropicResponsesRequest(ctx, request) + // Use a copy to avoid mutating caller-owned request state + reqCopy := *request + reqCopy.Model = deployment + reqBody, err := anthropic.ToAnthropicResponsesRequest(ctx, &reqCopy)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/providers/bedrock/anthropic_compat.go` around lines 83 - 85, The code currently mutates the incoming request by assigning request.Model = deployment before calling anthropic.ToAnthropicResponsesRequest, which can leak side effects; fix it by making a shallow copy of the request (e.g., copy := *request) and set copy.Model = deployment, then pass the copied value (or its pointer) into anthropic.ToAnthropicResponsesRequest so the original request remains unchanged.core/providers/bedrock/bedrock_test.go (1)
160-174: Prefer explicit scenario gating over commented-out test flags.Commenting these fields out silently disables a large test surface. Please set them explicitly (e.g., based on
s3Bucket/roleArn) so intent and coverage are clear.♻️ Proposed cleanup
- // BatchCreate: true, - // BatchList: true, - // BatchRetrieve: true, - // BatchCancel: true, - // BatchResults: true, - // FileUpload: true, - // FileList: true, - // FileRetrieve: true, - // FileDelete: true, - // FileContent: true, - // FileBatchInput: true, + BatchCreate: s3Bucket != "", + BatchList: s3Bucket != "", + BatchRetrieve: s3Bucket != "", + BatchCancel: s3Bucket != "", + BatchResults: s3Bucket != "", + FileUpload: s3Bucket != "", + FileList: s3Bucket != "", + FileRetrieve: s3Bucket != "", + FileDelete: s3Bucket != "", + FileContent: s3Bucket != "", + FileBatchInput: s3Bucket != "",🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/providers/bedrock/bedrock_test.go` around lines 160 - 174, The test flags in the Bedrock provider tests were left commented out (e.g., BatchCreate, BatchList, FileUpload, etc.), silently disabling many scenarios; update the test capability map where CountTokens, ImageEdit, ImageVariation, and StructuredOutputs are defined to explicitly set all relevant flags instead of commenting them out, using the existing environment checks (s3Bucket and roleArn) to gate features—for example, set BatchCreate, BatchList, BatchRetrieve, BatchCancel, BatchResults, FileUpload, FileList, FileRetrieve, FileDelete, FileContent, and FileBatchInput to true only when s3Bucket/roleArn indicate tests can run, and false otherwise so intent and coverage are explicit.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@core/providers/anthropic/responses.go`:
- Around line 158-162: The exported ReleaseAnthropicResponsesStreamState wrapper
currently calls releaseAnthropicResponsesStreamState which returns the object to
the pool without fully zeroing fields (e.g., CreatedAt left set and maps
reinitialized rather than nil), so update releaseAnthropicResponsesStreamState
(the underlying release path used by ReleaseAnthropicResponsesStreamState and
any callers) to explicitly reset all AnthropicResponsesStreamState fields to
their zero values before calling pool.Put(): set time fields like CreatedAt to
zero value, set pointer/slice/map fields to nil (not reinitialize), and clear
any buffers; ensure any necessary reinitialization happens in the acquire path
(where New/AcquireAnthropicResponsesStreamState prepares maps/buffers) rather
than on release.
In `@core/providers/bedrock/anthropic_compat.go`:
- Around line 120-125: MergeExtraParams can reintroduce reserved fields like
"model" and "stream" back into requestBody; after calling
providerUtils.MergeExtraParams(requestBody, extraParams) inside the passthrough
branch, remove or re-sanitize those reserved keys (e.g. delete
requestBody["model"] and requestBody["stream"] or call the existing sanitization
helper if one exists) so Bedrock/Anthropic requests cannot be invalidated;
ensure the re-sanitization happens immediately after the MergeExtraParams call
in the same block that checks
ctx.Value(schemas.BifrostContextKeyPassthroughExtraParams).
In `@core/providers/bedrock/bedrock.go`:
- Around line 1003-1004: The pooled Bedrock response objects are returned via
releaseBedrockChatResponse without clearing fields, causing stale-data leaks;
update releaseBedrockChatResponse to explicitly zero all fields of the object
referenced by bedrockResponse (and any nested structs/slices/maps) before
calling pool.Put(), and ensure all call sites (e.g., where
acquireBedrockChatResponse()/releaseBedrockChatResponse() are used at the shown
locations) continue to defer releaseBedrockChatResponse(bedrockResponse) after
using the response so the zeroing always runs.
- Around line 935-963: The routing logic currently calls
schemas.IsAnthropicModel(deployment) directly and can misclassify Anthropic
deployments when resolveBedrockDeployment returns an opaque alias/ARN; update
the Bedrock routing to compute a single isAnthropicRoute boolean (e.g. combine
schemas.IsAnthropicModel(deployment) with the existing model-based anthopic
check used elsewhere) right after resolveBedrockDeployment(request.Model, key)
and then use that isAnthropicRoute flag instead of calling
schemas.IsAnthropicModel(deployment) in the branches that set jsonData/path;
apply the same change in ChatCompletionStream, Responses, and ResponsesStream
(referencing resolveBedrockDeployment, schemas.IsAnthropicModel,
getBedrockAnthropicChatRequestBody, providerUtils.CheckContextAndGetRequestBody,
and provider.getModelPath) so anthopic/Claude traffic follows the invoke path
even when deployment is an alias.
---
Nitpick comments:
In `@core/providers/bedrock/anthropic_compat.go`:
- Around line 83-85: The code currently mutates the incoming request by
assigning request.Model = deployment before calling
anthropic.ToAnthropicResponsesRequest, which can leak side effects; fix it by
making a shallow copy of the request (e.g., copy := *request) and set copy.Model
= deployment, then pass the copied value (or its pointer) into
anthropic.ToAnthropicResponsesRequest so the original request remains unchanged.
In `@core/providers/bedrock/bedrock_test.go`:
- Around line 160-174: The test flags in the Bedrock provider tests were left
commented out (e.g., BatchCreate, BatchList, FileUpload, etc.), silently
disabling many scenarios; update the test capability map where CountTokens,
ImageEdit, ImageVariation, and StructuredOutputs are defined to explicitly set
all relevant flags instead of commenting them out, using the existing
environment checks (s3Bucket and roleArn) to gate features—for example, set
BatchCreate, BatchList, BatchRetrieve, BatchCancel, BatchResults, FileUpload,
FileList, FileRetrieve, FileDelete, FileContent, and FileBatchInput to true only
when s3Bucket/roleArn indicate tests can run, and false otherwise so intent and
coverage are explicit.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 46be53e5-47e1-42c4-98e4-056fde89fb20
📒 Files selected for processing (7)
core/changelog.mdcore/providers/anthropic/responses.gocore/providers/bedrock/anthropic_compat.gocore/providers/bedrock/bedrock.gocore/providers/bedrock/bedrock_test.gocore/providers/bedrock/types.gotransports/changelog.md
Comment on lines
+158
to
+162
| // ReleaseAnthropicResponsesStreamState returns an Anthropic responses stream state to the pool. | ||
| // Exported for use by providers that wrap Anthropic-compatible endpoints (e.g. Bedrock). | ||
| func ReleaseAnthropicResponsesStreamState(state *AnthropicResponsesStreamState) { | ||
| releaseAnthropicResponsesStreamState(state) | ||
| } |
Contributor
There was a problem hiding this comment.
Release wrapper now exposes a pool reset policy violation.
Line 161 delegates to a release path that does not reset all pooled fields to zero values before pool.Put() (for example, CreatedAt is set to current time and maps are reinitialized instead of zeroed). With this API exported, the risk footprint grows.
🔧 Suggested fix (zero on release, initialize on acquire)
func (state *AnthropicResponsesStreamState) flush() {
state.ChunkIndex = nil
state.AccumulatedJSON = ""
state.ComputerToolID = nil
state.WebSearchToolID = nil
state.WebSearchOutputIndex = nil
state.WebSearchResult = nil
- state.ContentIndexToOutputIndex = make(map[int]int)
- state.ContentIndexToBlockType = make(map[int]AnthropicContentBlockType)
- state.ToolArgumentBuffers = make(map[int]string)
- state.MCPCallOutputIndices = make(map[int]bool)
- state.ItemIDs = make(map[int]string)
- state.ReasoningSignatures = make(map[int]string)
- state.TextContentIndices = make(map[int]bool)
- state.ReasoningContentIndices = make(map[int]bool)
- state.CompactionContentIndices = make(map[int]*schemas.CacheControl)
+ state.ContentIndexToOutputIndex = nil
+ state.ContentIndexToBlockType = nil
+ state.ToolArgumentBuffers = nil
+ state.MCPCallOutputIndices = nil
+ state.ItemIDs = nil
+ state.ReasoningSignatures = nil
+ state.TextContentIndices = nil
+ state.ReasoningContentIndices = nil
+ state.CompactionContentIndices = nil
state.CurrentOutputIndex = 0
state.MessageID = nil
state.StopReason = nil
state.Model = nil
- state.CreatedAt = int(time.Now().Unix())
+ state.CreatedAt = 0
state.HasEmittedCreated = false
state.HasEmittedInProgress = false
state.StructuredOutputToolName = ""
state.StructuredOutputIndex = nil
}As per coding guidelines "Always reset all fields of pooled objects to their zero values before calling pool.Put(). Stale data from previous requests can leak to the next user of the pooled object."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@core/providers/anthropic/responses.go` around lines 158 - 162, The exported
ReleaseAnthropicResponsesStreamState wrapper currently calls
releaseAnthropicResponsesStreamState which returns the object to the pool
without fully zeroing fields (e.g., CreatedAt left set and maps reinitialized
rather than nil), so update releaseAnthropicResponsesStreamState (the underlying
release path used by ReleaseAnthropicResponsesStreamState and any callers) to
explicitly reset all AnthropicResponsesStreamState fields to their zero values
before calling pool.Put(): set time fields like CreatedAt to zero value, set
pointer/slice/map fields to nil (not reinitialize), and clear any buffers;
ensure any necessary reinitialization happens in the acquire path (where
New/AcquireAnthropicResponsesStreamState prepares maps/buffers) rather than on
release.
Comment on lines
+120
to
+125
| // Merge extra params if passthrough is enabled | ||
| if ctx.Value(schemas.BifrostContextKeyPassthroughExtraParams) != nil && ctx.Value(schemas.BifrostContextKeyPassthroughExtraParams) == true { | ||
| if len(extraParams) > 0 { | ||
| providerUtils.MergeExtraParams(requestBody, extraParams) | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
Re-sanitize reserved fields after passthrough merge.
MergeExtraParams runs after removing reserved keys, so model/stream can be reintroduced from extraParams. That can invalidate Bedrock Anthropic requests.
✅ Proposed fix
// Merge extra params if passthrough is enabled
if ctx.Value(schemas.BifrostContextKeyPassthroughExtraParams) != nil && ctx.Value(schemas.BifrostContextKeyPassthroughExtraParams) == true {
if len(extraParams) > 0 {
providerUtils.MergeExtraParams(requestBody, extraParams)
+ // Keep Bedrock Anthropic invariants intact after merge
+ delete(requestBody, "model")
+ delete(requestBody, "stream")
+ delete(requestBody, "fallbacks")
}
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| // Merge extra params if passthrough is enabled | |
| if ctx.Value(schemas.BifrostContextKeyPassthroughExtraParams) != nil && ctx.Value(schemas.BifrostContextKeyPassthroughExtraParams) == true { | |
| if len(extraParams) > 0 { | |
| providerUtils.MergeExtraParams(requestBody, extraParams) | |
| } | |
| } | |
| // Merge extra params if passthrough is enabled | |
| if ctx.Value(schemas.BifrostContextKeyPassthroughExtraParams) != nil && ctx.Value(schemas.BifrostContextKeyPassthroughExtraParams) == true { | |
| if len(extraParams) > 0 { | |
| providerUtils.MergeExtraParams(requestBody, extraParams) | |
| // Keep Bedrock Anthropic invariants intact after merge | |
| delete(requestBody, "model") | |
| delete(requestBody, "stream") | |
| delete(requestBody, "fallbacks") | |
| } | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@core/providers/bedrock/anthropic_compat.go` around lines 120 - 125,
MergeExtraParams can reintroduce reserved fields like "model" and "stream" back
into requestBody; after calling providerUtils.MergeExtraParams(requestBody,
extraParams) inside the passthrough branch, remove or re-sanitize those reserved
keys (e.g. delete requestBody["model"] and requestBody["stream"] or call the
existing sanitization helper if one exists) so Bedrock/Anthropic requests cannot
be invalidated; ensure the re-sanitization happens immediately after the
MergeExtraParams call in the same block that checks
ctx.Value(schemas.BifrostContextKeyPassthroughExtraParams).
Comment on lines
+1003
to
+1004
| bedrockResponse := acquireBedrockChatResponse() | ||
| defer releaseBedrockChatResponse(bedrockResponse) |
Contributor
There was a problem hiding this comment.
Pooled Bedrock response reuse now needs zero-before-put enforcement.
These paths add more reliance on releaseBedrockChatResponse, but that helper currently puts objects back without zeroing first.
Suggested fix (in helper)
func releaseBedrockChatResponse(resp *BedrockConverseResponse) {
- if resp != nil {
- bedrockChatResponsePool.Put(resp)
- }
+ if resp == nil {
+ return
+ }
+ *resp = BedrockConverseResponse{}
+ bedrockChatResponsePool.Put(resp)
}As per coding guidelines **/*.go: "Always reset all fields of pooled objects to their zero values before calling pool.Put(). Stale data from previous requests can leak to the next user of the pooled object."
Also applies to: 1470-1471
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@core/providers/bedrock/bedrock.go` around lines 1003 - 1004, The pooled
Bedrock response objects are returned via releaseBedrockChatResponse without
clearing fields, causing stale-data leaks; update releaseBedrockChatResponse to
explicitly zero all fields of the object referenced by bedrockResponse (and any
nested structs/slices/maps) before calling pool.Put(), and ensure all call sites
(e.g., where acquireBedrockChatResponse()/releaseBedrockChatResponse() are used
at the shown locations) continue to defer
releaseBedrockChatResponse(bedrockResponse) after using the response so the
zeroing always runs.
akshaydeo
changed the base branch from
03-04-fix_async_jobs_stuck_in_processing_on_marshal_failure_now_correctly_transition_to_failed_
to
graphite-base/1924
akshaydeo
force-pushed
the
graphite-base/1924
branch
from
50e1b1e to
bbf4084
Compare
akshaydeo
force-pushed
the
03-05-feat_shifted_to_anthropic_endpoints_in_bedrock_for_claude_models
branch
from
835a59c to
ba3d437
Compare
akshaydeo
force-pushed
the
03-05-feat_shifted_to_anthropic_endpoints_in_bedrock_for_claude_models
branch
from
ba3d437 to
4139d6f
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (3)
core/providers/bedrock/anthropic_compat.go (1)
120-125:⚠️ Potential issue | 🟠 MajorRe-sanitize reserved fields after passthrough merge.
MergeExtraParamscan reintroducemodel/stream/fallbacksafter they were stripped, which can invalidate Bedrock Anthropic InvokeModel payloads.✅ Proposed fix
// Merge extra params if passthrough is enabled if ctx.Value(schemas.BifrostContextKeyPassthroughExtraParams) != nil && ctx.Value(schemas.BifrostContextKeyPassthroughExtraParams) == true { if len(extraParams) > 0 { providerUtils.MergeExtraParams(requestBody, extraParams) + // Re-enforce Bedrock Anthropic invariants after merge + delete(requestBody, "model") + delete(requestBody, "stream") + delete(requestBody, "fallbacks") } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/providers/bedrock/anthropic_compat.go` around lines 120 - 125, After merging passthrough extra params (when ctx.Value(schemas.BifrostContextKeyPassthroughExtraParams) is true) re-sanitize the requestBody to remove any reserved fields that MergeExtraParams may have reintroduced (specifically "model", "stream", and "fallbacks"); locate the merge block around MergeExtraParams(requestBody, extraParams) in anthropic_compat.go and add logic after that call to explicitly delete or unset those keys from requestBody so the Bedrock Anthropic InvokeModel payload remains valid.core/providers/bedrock/bedrock.go (2)
941-942:⚠️ Potential issue | 🟠 MajorHarden Anthropic routing when deployment mappings are aliases/ARN-like identifiers.
Routing only on
schemas.IsAnthropicModel(deployment)can misclassify Anthropic traffic if deployment mapping is opaque. Use a unified predicate combining requested model + resolved deployment across all four flows.✅ Suggested pattern
deployment := resolveBedrockDeployment(request.Model, key) + isAnthropicRoute := schemas.IsAnthropicModel(request.Model) || schemas.IsAnthropicModel(deployment) - if schemas.IsAnthropicModel(deployment) { + if isAnthropicRoute { // Anthropic invoke path } else { // Converse path }Apply the same gate in
ChatCompletion,ChatCompletionStream,Responses, andResponsesStream.Also applies to: 1063-1064, 1408-1409, 1522-1523
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/providers/bedrock/bedrock.go` around lines 941 - 942, The Anthropic routing check is too fragile when deployment mappings are opaque; update each of the four flows (ChatCompletion, ChatCompletionStream, Responses, ResponsesStream) to use a unified predicate that checks both the requested model and the resolved deployment identifier (e.g., replace existing schemas.IsAnthropicModel(deployment) checks with something like schemas.IsAnthropicModel(requestedModel) || schemas.IsAnthropicModel(resolvedDeployment)); locate the branches that currently test schemas.IsAnthropicModel(deployment) (around the handlers for ChatCompletion, ChatCompletionStream, Responses, ResponsesStream) and change them to use the combined predicate so routing is correct for alias/ARN-like deployment mappings.
1003-1004:⚠️ Potential issue | 🟠 MajorPooled Bedrock responses still need zero-before-put enforcement.
These call paths now rely more on
releaseBedrockChatResponse, but that helper currently returns objects to the pool without clearing fields first.As per coding guidelines `**/*.go`: "Always reset all fields of pooled objects to their zero values before calling pool.Put(). Stale data from previous requests can leak to the next user of the pooled object."✅ Proposed fix in helper
func releaseBedrockChatResponse(resp *BedrockConverseResponse) { - if resp != nil { - bedrockChatResponsePool.Put(resp) - } + if resp == nil { + return + } + *resp = BedrockConverseResponse{} + bedrockChatResponsePool.Put(resp) }Also applies to: 1470-1471
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/providers/bedrock/bedrock.go` around lines 1003 - 1004, The pooled Bedrock response helper releaseBedrockChatResponse currently returns objects to the pool without clearing fields; update releaseBedrockChatResponse to explicitly reset every field of the pooled response struct to its zero value (strings to "", slices to nil/zero-length, maps to nil, pointers to nil, numeric fields to 0, boolean to false) before calling pool.Put, and ensure any other callers that return instances (e.g., paths using acquireBedrockChatResponse) rely on this zero-before-put guarantee so no stale data can leak into subsequent users.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@core/providers/bedrock/anthropic_compat.go`:
- Around line 83-85: The code mutates the caller-owned Responses request by
setting request.Model = deployment; instead avoid this side-effect by leaving
request unchanged and applying the resolved deployment to the converted
Anthropic request object returned by anthropic.ToAnthropicResponsesRequest(ctx,
request) (or by passing a shallow copy of request into the converter). Locate
the call site around anthropic.ToAnthropicResponsesRequest and ensure you either
pass a copy of request (so original ModelRequested is preserved) or set the
deployment on the resulting reqBody (e.g., reqBody.Model = deployment) rather
than mutating the original request.
---
Duplicate comments:
In `@core/providers/bedrock/anthropic_compat.go`:
- Around line 120-125: After merging passthrough extra params (when
ctx.Value(schemas.BifrostContextKeyPassthroughExtraParams) is true) re-sanitize
the requestBody to remove any reserved fields that MergeExtraParams may have
reintroduced (specifically "model", "stream", and "fallbacks"); locate the merge
block around MergeExtraParams(requestBody, extraParams) in anthropic_compat.go
and add logic after that call to explicitly delete or unset those keys from
requestBody so the Bedrock Anthropic InvokeModel payload remains valid.
In `@core/providers/bedrock/bedrock.go`:
- Around line 941-942: The Anthropic routing check is too fragile when
deployment mappings are opaque; update each of the four flows (ChatCompletion,
ChatCompletionStream, Responses, ResponsesStream) to use a unified predicate
that checks both the requested model and the resolved deployment identifier
(e.g., replace existing schemas.IsAnthropicModel(deployment) checks with
something like schemas.IsAnthropicModel(requestedModel) ||
schemas.IsAnthropicModel(resolvedDeployment)); locate the branches that
currently test schemas.IsAnthropicModel(deployment) (around the handlers for
ChatCompletion, ChatCompletionStream, Responses, ResponsesStream) and change
them to use the combined predicate so routing is correct for alias/ARN-like
deployment mappings.
- Around line 1003-1004: The pooled Bedrock response helper
releaseBedrockChatResponse currently returns objects to the pool without
clearing fields; update releaseBedrockChatResponse to explicitly reset every
field of the pooled response struct to its zero value (strings to "", slices to
nil/zero-length, maps to nil, pointers to nil, numeric fields to 0, boolean to
false) before calling pool.Put, and ensure any other callers that return
instances (e.g., paths using acquireBedrockChatResponse) rely on this
zero-before-put guarantee so no stale data can leak into subsequent users.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: e965c2c5-94e8-4ba4-871b-d536f842dff7
📒 Files selected for processing (7)
core/changelog.mdcore/providers/anthropic/responses.gocore/providers/bedrock/anthropic_compat.gocore/providers/bedrock/bedrock.gocore/providers/bedrock/bedrock_test.gocore/providers/bedrock/types.gotransports/changelog.md
🚧 Files skipped from review as they are similar to previous changes (4)
- core/providers/anthropic/responses.go
- core/providers/bedrock/bedrock_test.go
- core/changelog.md
- transports/changelog.md
Comment on lines
+83
to
+85
| // Mutate the model before conversion so converters see the resolved deployment name | ||
| request.Model = deployment | ||
| reqBody, err := anthropic.ToAnthropicResponsesRequest(ctx, request) |
Contributor
There was a problem hiding this comment.
Avoid mutating the inbound Responses request model.
request.Model = deployment introduces side effects on the caller-owned object and can corrupt ModelRequested metadata later in the request flow. Set the deployment on the converted Anthropic request body instead.
✅ Proposed fix
- // Mutate the model before conversion so converters see the resolved deployment name
- request.Model = deployment
reqBody, err := anthropic.ToAnthropicResponsesRequest(ctx, request)
if err != nil {
return nil, providerUtils.NewBifrostOperationError(schemas.ErrRequestBodyConversion, err, providerName)
}
if reqBody == nil {
return nil, providerUtils.NewBifrostOperationError("request body is not provided", nil, providerName)
}
+ reqBody.Model = deployment📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| // Mutate the model before conversion so converters see the resolved deployment name | |
| request.Model = deployment | |
| reqBody, err := anthropic.ToAnthropicResponsesRequest(ctx, request) | |
| reqBody, err := anthropic.ToAnthropicResponsesRequest(ctx, request) | |
| if err != nil { | |
| return nil, providerUtils.NewBifrostOperationError(schemas.ErrRequestBodyConversion, err, providerName) | |
| } | |
| if reqBody == nil { | |
| return nil, providerUtils.NewBifrostOperationError("request body is not provided", nil, providerName) | |
| } | |
| reqBody.Model = deployment |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@core/providers/bedrock/anthropic_compat.go` around lines 83 - 85, The code
mutates the caller-owned Responses request by setting request.Model =
deployment; instead avoid this side-effect by leaving request unchanged and
applying the resolved deployment to the converted Anthropic request object
returned by anthropic.ToAnthropicResponsesRequest(ctx, request) (or by passing a
shallow copy of request into the converter). Locate the call site around
anthropic.ToAnthropicResponsesRequest and ensure you either pass a copy of
request (so original ModelRequested is preserved) or set the deployment on the
resulting reqBody (e.g., reqBody.Model = deployment) rather than mutating the
original request.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrated Bedrock provider to use Anthropic's native Messages API format for Claude models instead of the Bedrock Converse API. This change improves compatibility and leverages Anthropic's native streaming format while maintaining backward compatibility for non-Anthropic models.
Changes
anthropic_compat.gofile with functions to convert requests to Anthropic Messages API format for Bedrock's InvokeModel endpointType of change
Affected areas
How to test
Test with Claude models on Bedrock to verify native Anthropic API compatibility:
Verify non-Anthropic models still use Converse API and function correctly.
Screenshots/Recordings
N/A
Breaking changes
This change maintains backward compatibility. Existing Bedrock integrations will continue to work without modification.
Related issues
N/A
Security considerations
No new security implications. Uses existing Bedrock authentication and maintains the same security model.
Checklist
docs/contributing/README.mdand followed the guidelines