Skip to content

Add LiteLLM support #4078

Description

@ocs-percy

Add LiteLLM Support

Integrate LiteLLM as a supported LLM provider/proxy in Open Chat Studio.

LiteLLM provides a unified interface to 100+ LLMs and can be self-hosted as a proxy, making it popular for teams that want to centralise model access, add spend controls, and maintain a consistent API surface regardless of underlying provider.

What needs to happen

  • Add LiteLLM (proxy) as a provider option in the model configuration UI
  • Support configuring a custom LiteLLM base URL and API key per team/org
  • Map available models from the LiteLLM /models endpoint (or allow manual entry)
  • Handle LiteLLM-specific behaviours (e.g. streaming differences, error formats)
  • Test against a self-hosted LiteLLM instance with multiple backend models

Notes

LiteLLM exposes an OpenAI-compatible API, so this may share a code path with the OpenRouter integration. Consider whether both can be handled under a generic "OpenAI-compatible endpoint" provider type.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status
    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions