Skip to content

Conversation

jamessanford
Copy link
Contributor

@jamessanford jamessanford commented Aug 6, 2025

  • Add tool support for OpenRouter by propagating tool support from the supported_parameters field in the models API output
  • Remove unnecessary second fetch of structured_outputs.
  • Update recorded HTTP API interactions and snapshots.
  • Add tool support test.

This resolves:

Note that streaming tool calls to certain models may still fail in version of llm up to 0.26 [edit: fixed in llm 0.27] due to:

Note:

This is an alternate, smaller PR equivalent to #42. Compared to #42, the cassette recordings now only include a subset of models supported by OpenRouter, instead of the entire 400k response.

@jamessanford jamessanford force-pushed the tool_support_smaller_cassettes branch from 2228895 to 5443a2c Compare August 11, 2025 22:57
@jamessanford jamessanford changed the title Add tool support and tests (smaller cassettes than #42) Add tool support and tests Aug 11, 2025
To avoid error about supports_tools= parameter in constructor
@simonw
Copy link
Owner

simonw commented Sep 20, 2025

Testing this with:

llm -m openrouter/openai/gpt-5 'current version' -T llm_version --td

It output:


Tool call: llm_version({})
  0.27.1

LLM version: 0.27.1

@simonw
Copy link
Owner

simonw commented Sep 20, 2025

And a more complex example:

llm install llm-tools-datasette
llm chat -m openrouter/openai/gpt-5 --td -T 'Datasette("https://datasette.io/content")'

Then in the chat:

Show tables

Then:

Show a count of rows in each of the views

Which worked - transcript here: https://gist.github.com/simonw/4b7c889ac6ed22b7d691f4981dedbe05

@simonw
Copy link
Owner

simonw commented Sep 20, 2025

@jamessanford this is excellent, thanks very much for this.

@simonw simonw merged commit 603cd08 into simonw:main Sep 20, 2025
5 checks passed
This was referenced Sep 20, 2025
@simonw
Copy link
Owner

simonw commented Sep 20, 2025

Here's a model that does not support tools:

llm -m openrouter/nousresearch/hermes-2-pro-llama-3-8b hi -T llm_version

Output:

Error: OpenRouter: openrouter/nousresearch/hermes-2-pro-llama-3-8b does not support tools

So that works as it should.

simonw added a commit that referenced this pull request Sep 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants