feat(langchain/createAgent): Add anthropicCodeExecution middleware #1
Conversation
pokey
force-pushed
the
pokey/anthropic-code-execution-middleware
branch
from
5358b8c to
7ed4fbf
Compare
There was a problem hiding this comment.
Not sure where this file (and its corresponding test) should go. Tbh most of the functions we probably don't want to support ourselves, but extractGeneratedFilesAnthropic should maybe go into anthropic provider? cc/ @hntrl
There was a problem hiding this comment.
I wonder if it makes sense to collapse this as gated logic inside of the anthropicCodeExecution? E.g. when we specify middleware we can attach files and those will be auto-uploaded to anthropic, and file contents + metadata will be contained within the content of the message
Would agree that it doesn't make a lot of sense to expose these as user APIs
| * import { | ||
| * uploadFileAnthropic, | ||
| * extractGeneratedFilesAnthropic, | ||
| * getFileMetadataAnthropic, | ||
| * downloadFileAnthropic, | ||
| * } from "langchain/agents/middleware"; |
There was a problem hiding this comment.
note that if we don't end up supporting these, we should update the jsdoc to just inline their impls; they're fairly simple
| * @see {@link uploadFileAnthropic} for uploading files to Anthropic | ||
| * @see {@link extractGeneratedFilesAnthropic} for extracting generated file IDs from responses | ||
| * @see {@link getFileMetadataAnthropic} for retrieving file metadata including filenames | ||
| * @see {@link downloadFileAnthropic} for downloading generated files from Anthropic |
| messages: workAroundAnthropicCodeExecutionBug(request.messages), | ||
| callOptions: { | ||
| // Pass container ID to reuse files across turns | ||
| // FIXME: What to do about container expiration? |
There was a problem hiding this comment.
wasn't sure how to handle container expiration. Open to suggestions, including punt
| * @param messages The messages to filter | ||
| * @returns The filtered messages | ||
| */ | ||
| function workAroundAnthropicCodeExecutionBug( |
There was a problem hiding this comment.
This is unfortunate. I've sent an email to anthropic support with a link to a gist for them to reproduce it
There was a problem hiding this comment.
we can flag this with them internally also!
| /** | ||
| * Anthropic code execution content block types. | ||
| * These match the types from @anthropic-ai/sdk BetaBashCodeExecution* interfaces. | ||
| */ |
There was a problem hiding this comment.
Not ideal to reproduce all these types here but I didn't want to depend on our anthropic provider or anthropic sdk
There was a problem hiding this comment.
good call, makes for even more reason to have integration specific middleware!
There was a problem hiding this comment.
Wasn't sure where to put this. Should maybe go somewhere in the docs?
There was a problem hiding this comment.
examples/ is typically where we dump stuff like this
| "format:check": "prettier --config .prettierrc --check \"src\"" | ||
| }, | ||
| "devDependencies": { | ||
| "@anthropic-ai/sdk": "^0.56.0", |
There was a problem hiding this comment.
we need to remove this once we figure out where the anthropic file helpers should go; left here for now so people can play with the code
There was a problem hiding this comment.
this goes with the dataAnalysisExperiment; we should remove once we figure out where that is going
There was a problem hiding this comment.
Fwiw here is an example run:
- First step: https://smith.langchain.com/public/51287a64-9011-4c0b-bb81-2666d694b0d0/r
- Second step: https://smith.langchain.com/public/05a8176a-2b4f-4e97-9639-3a6dd1f3ed4a/r
I was kinda hoping because they share thread id they'd end up in same run?
There was a problem hiding this comment.
There is a "Threads" tab in a tracing project that should let you view both (though you can't share that thread publicly)
| for (const fileId of extractGeneratedFilesAnthropic(response2)) { | ||
| const metadata = await getFileMetadataAnthropic(client, fileId); | ||
| await downloadFileAnthropic(client, fileId, metadata.filename); | ||
| console.log(`Downloaded generated file: ${metadata.filename}`); | ||
| } |
There was a problem hiding this comment.
would be cool if these files could make it into langsmith trace
There was a problem hiding this comment.
I wonder if we want any tracking of uploaded / generated files? The latter might be useful
There was a problem hiding this comment.
how do you imagine this looking? is this not retained in the returned message history? (is that another content type that we need to add?)
|
One thing I've left completely unexplored and untested is how this machinery interacts if you also have local tools |
| * @param messages The messages to filter | ||
| * @returns The filtered messages | ||
| */ | ||
| function workAroundAnthropicCodeExecutionBug( |
There was a problem hiding this comment.
FWIW I could also see this code migrating to anthropic message parser
|
Closing in favour of langchain-ai#9209 |
Adds middleware to enable code execution using Anthropic's beta code execution tool. The middleware does the following:
This PR also includes a simple standalone example usage file, though I'm not entirely sure where to put that
Depends on:
modelSettingsto middleware langchain-ai/langchainjs#9110