Replies: 6 comments
|
I would break this into three separate pieces: identity propagation, upstream MCP auth, and MCP tool-call cost accounting. 1. Make sure LiteLLM can identify the end userFor per-user MCP auth to work, the LiteLLM key/session needs a stable user identity. In practice, make sure OpenWebUI is not calling LiteLLM as one anonymous/shared service key only. You want the request/key context to include a real 2. Use per-user OAuth for upstream MCP, not one shared credentialRecent LiteLLM code has support for per-user MCP OAuth credential storage and status endpoints, for example endpoints like: The MCP server table/model also has fields such as
Since you are on 3. Cost tracking exists, but only if you provide a cost modelLiteLLM has an MCP cost calculator path for tool calls. From the current code, MCP cost defaults to {
"default_cost_per_query": 0.01,
"tool_name_to_cost_per_query": {
"dataverse_search": 0.02,
"copilot_rag_query": 0.05
}
}That means LiteLLM can track a fixed per-call cost per MCP server/tool. If Microsoft Dataverse/Copilot returns dynamic usage-based cost, you will likely need a hook after the MCP tool call to set Suggested minimal test
So: yes, the architecture you want is the right direction, but I would not expect it to be just header forwarding from OpenWebUI. Treat LiteLLM as the identity + credential broker: stable end-user identity in, per-user OAuth credential stored for the MCP server, then fixed or hook-derived MCP tool cost attached to the tool call. |
|
@Gloria72's three-way split is right; here's how it maps onto LiteLLM specifically:
Happy to go deeper on the callback shape — I've worked in litellm's codebase. Short version: don't expect built-in per-user OAuth brokering or per-call MCP pricing; plan for token passthrough + a custom cost callback. |
|
For this setup, I would separate three identities instead of trying to solve everything with one credential:
For 400 users, I would avoid one shared OAuth credential for the MCP server if the MCP server accesses user-owned resources such as Microsoft 365. A safer production pattern is: The important separation is:
For cost tracking, I would attach stable metadata to every request/tool call: {
"user_id": "user_123",
"team_id": "team_abc",
"mcp_server": "microsoft365",
"tool_name": "search_mail",
"request_id": "req_123"
}Then cost can be attributed at the LiteLLM layer even if the upstream MCP server has its own OAuth flow. If LiteLLM’s current MCP OAuth credential endpoints are available in your version, I would first check whether the user credential is being stored and retrieved per The main thing to verify is that the OAuth token is used for subsequent MCP tool calls, not only during the initial connection flow. |
|
@almajo
LiteLLM has built-in MCP APIs for managing per-user credentials, including OAuth-backed servers:
These APIs are designed to associate credentials with a specific user and MCP server, rather than using one shared credential for everyone. :contentReference[oaicite:0]{index=0} For your setup, I'd recommend: This gives you:
Regarding cost tracking, LiteLLM's MCP gateway supports assigning costs to MCP tool calls, but those costs aren't inferred automatically from third-party services. If your upstream service has a fixed per-call price (e.g. Dataverse API calls), you can configure that. If the service returns dynamic billing information, you'd likely need to capture that in a callback or custom integration and attach it to the request before logging. :contentReference[oaicite:1]{index=1} One thing I'd check is your LiteLLM version. You mentioned 1.88.1—the MCP OAuth APIs and user-credential endpoints are present in the current API surface, so if you don't see those endpoints in your deployment, upgrading would be worthwhile before investing much effort in the integration. :contentReference[oaicite:2]{index=2} A couple of questions that would help narrow down the best approach:
The answer changes slightly depending on who owns the OAuth lifecycle. If this solves your problem, feel free to mark it as the accepted answer so others can find it easily. |
|
Disclosure: TrustGate / NeuralTrust DevRel. For Open WebUI (N users) → gateway → OAuth MCP servers with per-call cost, the hard part is keeping one user identity all the way through — not just proxying tools. Patterns that usually work:
LiteLLM's MCP proxy is a reasonable place to host (2)+(4) if you're already standardized on it. If you outgrow LLM-only tenancy and want the same consumers/audit across chat completions and MCP, that's the Agent Gateway shape we work on with TrustGate (Go, Apache-2.0): https://github.com/NeuralTrust/TrustGate — either way, fix identity propagation before tuning cost dashboards. |
|
In LiteLLM with OpenWebUI for multi-tenant MCP tool routing and user-specific cost tracking:
model_list:
- model_name: mcp-tool-router
litellm_params:
model: mcp/copilot-rag
api_base: https://your-mcp-service.internal
forward_headers: ["Authorization", "X-User-Id"]
general_settings:
track_user_cost: true
|
Uh oh!
There was an error while loading. Please reload this page.
I have the following target setup:
As some of our MCP servers that we use have a cost-per-call (like Microsoft Dataverse or Copilot RAG API), we would love to track that cost in LiteLLM by adding it as a Litelllm MCP server.
However, I have not found the right settings of how authentication flows from OpenWebUI through the litellm-proxy to the target-mcp and we can still track the tool-calling-costs.
Has anybody managed to implement this in such a setting? After all it's the main use case with mcp servers, where multiple users call the same MCP server in their name.
Really happy to hear back for this. I'm currently on litellm version 1.88.1
All reactions