Skip to content

[BUG] OpenAi tool_choice being overridden when using with_params #317

@imtyM

Description

@imtyM

Basic checks

  • I searched existing issues - this hasn't been reported
  • I can reproduce this consistently
  • This is a RubyLLM bug, not my application code

What's broken?

Setting a custom tool choice param for OpenAi is being overridden and set to tool_choice: "auto" - preventing the possibility of using the tool_choice: "required" feature.
This is due to

if tools.any?
payload[:tools] = tools.map { |_, tool| tool_for(tool) }
payload[:tool_choice] = 'auto'
end

This is related to #303 ("dangerously" setting params), but I think the tool choice in render_payload should be removed?

The open ai docs suggest that it would default to auto if tools are present.

How to reproduce

  1. Configure RubyLLM using OpenAi and a model that supports tools e.g. gpt-4o-mini
  2. Enable debug to see the request export RUBYLLM_DEBUG=true
  3. create and set a tool to the chat instance
  4. set a custom tool_choice: 'required' param
  5. ask.

Expected behavior

tool_choice should be "required" in the payload.

What actually happened

tool_choice is set to "auto"

Environment

Ruby version: 3.2.3
RubyLLM version: 1.4.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions