What's the best way to keep the model registry up-to-date?

[danielfriis]

I know about RubyLLM.models.refresh! and .save_to_json, but for some reason it doesn't seem to persist. Very often when I push new versions to production, I get an error that newer models aren't recognised

[crmne]

Did you see any errors from refreshing or saving?

[toddkummer]

I ran into this issue as well. I was working in a Rails application and for me the root cause was not understanding the relationship between the JSON file, what's in memory, and the database. Running the rake task pulls what is in the JSON file into the database, but does not necessarily include new models if the JSON file is out of date.

In the end I skipped the JSON file and updated the database from memory:

RubyLLM::Models.refresh!
model_class = RubyLLM.config.model_registry_class.constantize
model_class.save_to_database

Perhaps that logic could be added as an additional rake task. It is a bit YOLO-ish, since you don't always know what is going to get added and it would be required to be run in each environment.

An alternative might be to recommend configuring the JSON file to be stored in the seeds directory and letting it be versioned. Then a seed file could be added that essentially runs the load_models rake task. Deployments would keep environments in sync and models would be added in a consistent, tracked manner.

[AlexVPopov]

We've done it the following way:

• we don't use the Rails integration, so we've opted for a file-based registry:

RubyLLM.configure do |config|
  config.model_registry_file = Rails.root.join("config/ruby_llm/models.json").to_s
end

• in config/initializers/ruby_llm.rb:

RubyLLM.models.load_from_json!(RubyLLM.config.model_registry_file)

• to keep the file up-to-date, we've setup a recurring GitHub Action, which:
  • runs bundle exec rake ruby_llm:refresh_model_registry;
  • checks if there's a diff in model.json;
  • if there is - it opens a pull request.