[BUG] Ollama Wouldn’t Get Configured as the Only Provider

[o0nj]

Basic checks

• I searched existing issues - this hasn't been reported
• I can reproduce this consistently
• This is a RubyLLM bug, not my application code

What's broken?

Ollama cannot be configured as the sole provider as it depends on the OpenAI provider.

Could be because the Ollama provider extends on Ollama, which in turn requires an API key, at

ruby_llm/lib/ruby_llm/providers/ollama.rb

Line 7 in 0924d2e

extend OpenAI

RubyLLM::Models.fetch_from_providers

[]

How to reproduce

On a fresh installation,

RubyLLM.configure do |config|
  config.ollama_api_base = "http://localhost:11434/v1"
end

RubyLLM.chat.ask

Expected behavior

Chat response.

What actually happened

RubyLLM::ConfigurationError: openai provider is not configured. Add this to your initialization: (RubyLLM::ConfigurationError)

Environment

• Ruby (3.4.3)
• ruby_llm (1.31)
• With the sole provider, Ollama.

[compumike]

Hi @o0nj, I'm not sure this is a bug. You're seeing that message because the default for config.default_model = 'gpt-4.1-nano', which resolves via models.json to be an :openai model.

ruby_llm/lib/ruby_llm/configuration.rb

Line 56 in 5a55d90

@default_model = 'gpt-4.1-nano'

With Ollama, RubyLLM does not initially know which models are available, because they're not baked into models.json https://github.com/crmne/ruby_llm/blob/main/lib/ruby_llm/models.json . Partially that's because with ollama, it's local and dependent on which models you've installed, so there's no globally published list of available models.

For both solutions below, I'll assume you've already done ollama pull llama3.2:1b.

Easy solution (recommended!)

Just specify the model and provider each time you create a chat:

require 'ruby_llm'

RubyLLM.configure do |config|
  config.ollama_api_base = "http://localhost:11434/v1"
end

chat = RubyLLM.chat(model: "llama3.2:1b", provider: :ollama)
response = chat.ask("What is 2+2? Answer with the number only.")
response.content
=> "4"

Note that you'll also see a warning from RubyLLM like this:

WARN -- RubyLLM: Assuming model 'llama3.2:1b' exists for provider 'RubyLLM::Providers::Ollama'. Capabilities may not be accurately reflected.

but that's okay!

Advanced, but hacky solution

Pull the list from the Ollama server, and replace the models list with it.

require 'ruby_llm'

RubyLLM.configure do |config|
  config.ollama_api_base = "http://localhost:11434/v1"

  # Retrieve available models from the Ollama server
  ollama_provider = RubyLLM::Provider.providers[:ollama]
  ollama_models = ollama_provider.list_models(connection: ollama_provider.connection(RubyLLM.config))

  # Replace RubyLLM.models list with the Ollama models only
  # (Note: very hacky use of RubyLLM.models.all to mutate the array!)
  RubyLLM.models.all.clear
  RubyLLM.models.all.concat(ollama_models)

  # Print out which models are available
  model_ids = RubyLLM.models.all.map(&:id)
  RubyLLM.logger.info "Loaded models list from Ollama server. Available models: #{model_ids}"

  # Set a default model
  config.default_model = "llama3.2:1b"
end

chat = RubyLLM.chat
response = chat.ask("What is 2+2? Answer with the number only.")
response.content
=> "4"

This is complicated and dependent on mutating an array which wasn't really meant to be mutated, so I wouldn't really recommend it. But it's a possibility, and maybe even should get added to RubyLLM as a method. :)

If I were you, I'd just do the "easy solution" above.

[o0nj]

Closing with a related contribution,

Allow local providers to fetch model list #294

[crmne]

It's now actually fixed c798135