Feature - Portkey tracing integration

[priyankinfinnov]

Pull Request

Summary

Added tracing support for Portkey AI Gateway, enabling automatic monitoring of all LLM completions routed through Portkey's unified interface. This integration allows users to trace and monitor completions across 100+ LLM providers supported by Portkey through a single interface, with full support for both streaming and non-streaming completions.

Changes

• Implement portkey_tracer.py with instance-level patching of Portkey's chat.completions.create method
• Add support for streaming and non-streaming completion tracing
• Implement provider detection from Portkey headers, response metadata, and model name heuristics
• Extract Portkey-specific metadata (trace IDs, cache status, retry counts)
• Add usage and cost calculation with fallback estimation for streaming completions
• Implement metadata extraction that works across different Portkey SDK versions
• Add comprehensive test suite covering all major functionality (test_portkey_integration.py)
• Export trace_portkey() function in lib/__init__.py for public API
• Add example notebook demonstrating Portkey tracing usage (portkey_tracing.ipynb)

Context

Portkey AI Gateway provides a unified interface to call 100+ LLM APIs using the same OpenAI-compatible input/output format. This integration enables Openlayer users to monitor and trace all their LLM completions routed through Portkey, regardless of the underlying provider (OpenAI, Anthropic, Google, Meta, Mistral, Cohere, etc.).

The implementation follows a similar pattern to the existing LiteLLM integration but uses instance-level patching (via __init__ wrapping) to handle Portkey's lazy initialization of the chat attribute. This approach ensures compatibility across different Portkey SDK versions and handles edge cases gracefully.

Key features:

• Automatic provider detection from Portkey headers or response metadata
• Portkey-specific metadata capture (trace IDs, cache hits/misses, retry attempts)
• Sensitive header redaction (API keys, virtual keys)
• Support for tool calls and function calls
• Cost calculation with provider-specific heuristics

Testing

• Unit tests
• Manual testing

Unit Tests: Comprehensive test suite with 20+ test cases covering:

• Import handling when Portkey is unavailable
• Instance-level patching behavior
• Streaming and non-streaming completion handling
• Provider detection (headers, metadata, model name fallbacks)
• Metadata extraction (client headers, response headers, unit metadata)
• Usage extraction from responses and chunks
• Cost and token calculation for streaming completions
• Output parsing (content, function calls, tool calls)
• Trace argument creation and provider metadata passing

Manual Testing: Verified with example notebook demonstrating:

• Basic completion tracing
• Tool calling with function definitions
• Automatic metadata capture
• Provider detection across different models

After merge

• We must update public documentation to add a page for Portkey Integration with appropriate details, logo, examples and links etc.