What's the best way to keep the model registry up-to-date?

[danielfriis]

I know about RubyLLM.models.refresh! and .save_to_json, but for some reason it doesn't seem to persist. Very often when I push new versions to production, I get an error that newer models aren't recognised

[crmne]

Did you see any errors from refreshing or saving?

[toddkummer]

I ran into this issue as well. I was working in a Rails application and for me the root cause was not understanding the relationship between the JSON file, what's in memory, and the database. Running the rake task pulls what is in the JSON file into the database, but does not necessarily include new models if the JSON file is out of date.

In the end I skipped the JSON file and updated the database from memory:

RubyLLM::Models.refresh!
model_class = RubyLLM.config.model_registry_class.constantize
model_class.save_to_database

Perhaps that logic could be added as an additional rake task. It is a bit YOLO-ish, since you don't always know what is going to get added and it would be required to be run in each environment.

An alternative might be to recommend configuring the JSON file to be stored in the seeds directory and letting it be versioned. Then a seed file could be added that essentially runs the load_models rake task. Deployments would keep environments in sync and models would be added in a consistent, tracked manner.

[AlexVPopov]

We've done it the following way:

• we don't use the Rails integration, so we've opted for a file-based registry:

RubyLLM.configure do |config|
  config.model_registry_file = Rails.root.join("config/ruby_llm/models.json").to_s
end

• in config/initializers/ruby_llm.rb:

RubyLLM.models.load_from_json!(RubyLLM.config.model_registry_file)

• to keep the file up-to-date, we've setup a recurring GitHub Action, which:
  • runs bundle exec rake ruby_llm:refresh_model_registry;
  • checks if there's a diff in model.json;
  • if there is - it opens a pull request.

[epugh]

What are the pros of using the database and the Models instead of just using RubyLLM.models? My Rails app https://github.com/o19s/quepid gets installed by many companies internally, so I want this to be as simple as possible.

Is just having RubyLLM.models.refresh! in my config/initializrs/ruby_llm.rb not good enough? That way, everytime they do a fresh update of my app (which happens every few months) then they get the updated list?

[github-actions[bot]]

If you've run the install generator, RubyLLM.models.refresh already persists into RubyLLM's own model table, so an initializer call keeps every environment in sync on each deploy without a JSON file. The database registry also tracks "unlisted" models so chats keep working even if a model disappears upstream. If you're not using the Rails integration, RubyLLM.models.refresh alone (no JSON) is fine too, though you lose the unlisted-model safety net. See https://rubyllm.com/models/ for details.

Generated by Copilot issue assessment · copilot · auto · 17.9 AIC · ⌖ 6.68 AIC · ⊞ 8.6K · ◷