OpenRouter model from API automatically #13721
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
|
|
||
| # OpenRouter has an API endpoint to get model info | ||
| if custom_llm_provider == "openrouter": | ||
| model_info = _get_model_info_open_router(model_name=model.strip(custom_llm_provider + "/")) |
There was a problem hiding this comment.
it's dangerous to add a sync network call to a helper util which can be used across the repo.
It would be safer to allow a user to run a function - e.g. litellm.pull_openrouter_model_info() -> and call litellm.register_model() in there - to add the values to the model cost map.
this way litellm.get_model_info will just work, without any network calls.
There was a problem hiding this comment.
That's a very good point, I had a hard time figure out where this method should live, and I saw we already have similar code for huggingface.
I'll make those change as soon as I have the time.
There was a problem hiding this comment.
The other possibility would be to fix : https://github.com/BerriAI/litellm/actions/workflows/auto_update_price_and_context_window.yml
This github action, I could make it that it commit directly to the main branch by itself on a daily basis if there are changes.
Here is the PR for that, it would be more aligned with the project:
#13816
|
I've gone ahead and instead decided to update the GitHub action and the python script that scrape the data of open router models: |
|
@krrishdholakia I think #13816 is a better solution since we rely on updating automatically the json file with model info. |
Title
Get the model information of OpenRouter model automatically
Relevant issues
Fixes #13653
Partially, I couldn't find the way to have LiteLLM use the usage.cost parameter to return the actual cost of the request.
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitType
🆕 New Feature
Changes
Make the mode_info provider use the OpenRouter API to get missing data automatically.