prompt caching SPIKE #109
prompt caching SPIKE #109
Conversation
| @@ -13,6 +13,8 @@ | |||
| url: /guides/tools | |||
| - title: Streaming | |||
| url: /guides/streaming | |||
| - title: Prompt Caching | |||
There was a problem hiding this comment.
Think Cline got a little overzealous w/ documentation
| @@ -67,6 +67,29 @@ def supports_json_mode?(model_id) | |||
| def supports_extended_thinking?(model_id) | |||
| model_id.match?(/claude-3-7-sonnet/) | |||
| end | |||
|
|
|||
There was a problem hiding this comment.
I imagine there is an existing philosophy how to use this data I am unaware of
| @@ -26,16 +26,22 @@ def initialize(model: nil, provider: nil) | |||
| end | |||
|
|
|||
| def ask(message = nil, with: {}, &block) | |||
| add_message role: :user, content: Content.new(message, with) | |||
| # Extract cache_control from the with hash if present | |||
There was a problem hiding this comment.
I'm unsure if this is how we want to handle things. this essentailly adds global functionality for an anthropic specific concept.
| cache_control: true) | ||
|
|
||
| # Add a PDF document with cache control | ||
| chat.ask("Please analyze this document:", |
There was a problem hiding this comment.
this API seems fine to me. We need to tag the messages with "this is where to cache", so the with option seems like a fine place for that.
|
What's a \prompt caching SPIKE? |
|
It makes things hella cheap https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching Some models do it automatically, some don’t I’m unsure if I’m editing the library code in a sensible way, what do you think? |
|
Ok, cool. Can you please change the title and description of your PR? |
No description provided.