Anthropic: Fix system prompt (use plain text instead of serialized JSON)

[MichaelHoste]

What this does

When migrating from ruby-openai, I had some issues getting the same responses in my Anthropic test suite.

After some digging, I observed that the Anthropic requests send the system context as serialized JSON instead of a plain string like described in the API reference:

{
  :system => "{type:\n        \"text\", text: \"You must include the exact phrase \\\"XKCD7392\\\" somewhere\n        in your response.\"}",
  [...]
}

instead of :

{
  :system => "You must include the exact phrase \"XKCD7392\" somewhere in your response.",
  [...]
}

It works quite well (the model still understands it) but it uses more tokens than needed. It could also mislead the model in interpreting the system prompt.

This PR fixed it. I also took the initiative to make the temperature an optional parameter (just like with OpenAI). I hope it's not too much for a single PR, but since I was already re-recording the cassettes, I figured it would be easier.

I'm sorry but I don't have any API key for Bedrock/OpenRouter. I only recorded the main Anthropic cassettes.

Type of change

• Bug fix
• New feature
• Breaking change
• Documentation
• Performance improvement

Scope check

• I read the Contributing Guide
• This aligns with RubyLLM's focus on LLM communication
• This isn't application-specific logic that belongs in user code
• This benefits most users, not just my specific use case

Quality check

• I ran overcommit --install and all hooks pass
• I tested my changes thoroughly
• I updated documentation if needed
• I didn't modify auto-generated files manually (models.json, aliases.json)

API changes

• Breaking change
• New public methods/classes
• Changed method signatures
• No API changes

Related issues

[tpaulshippy]

Just a note that #234 also fixes this issue but in a slightly different way. https://docs.anthropic.com/en/api/messages#body-system shows that you can provide a string or an array of objects. When caching system prompts, you have to use the array to provide the cache_control attribute, so I switched to providing the array instead of the string. Without that PR though, this PR will definitely clean up the string.

[MichaelHoste]

Thanks @tpaulshippy !

If #234 solves the issue and is merged soon, it's good to me 🙂 I noticed in the cassettes that the JSON was not serialized anymore so that should be ok.

But since it's a bug, if #234 still need some time to be merged, this one could maybe be merged first? There will not be a lot of conflicts. And I still think that the temperature should be an optional parameter for Anthropic (same as OpenAI).

[tpaulshippy]

Agreed. Would love to see this get merged ASAP.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.61%. Comparing base (8c9d990) to head (852b330).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #302   +/-   ##
=======================================
  Coverage   86.61%   86.61%           
=======================================
  Files          79       79           
  Lines        3123     3124    +1     
  Branches      611      612    +1     
=======================================
+ Hits         2705     2706    +1     
  Misses        418      418           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[crmne]

LGTM