Emit privacy-safe OpenTelemetry traces from AI Gateway and vLLM

[ns-mkusper]

Problem

The model-serving pack does not configure Envoy AI Gateway or vLLM to emit session-correlated OpenTelemetry traces. Operators can see aggregate metrics, but cannot follow one chat-client request through gateway routing, queueing, inference, and streaming completion.

Scope

• Add an opt-in tracing contract for the pack/operator that configures:
  • Envoy AI Gateway OTLP export and a stable service name.
  • x-session-id → session.id span mapping and W3C trace-context propagation.
  • vLLM --otlp-traces-endpoint plus OTLP protocol/service resource attributes.
• Hide OpenInference inputs, outputs, embedding text, and vectors by default.
• Keep vLLM detailed model/worker traces disabled by default because they can affect performance.
• Support a configured cluster OpenTelemetry Collector endpoint instead of deploying another collector.
• Add an E2E test with a local OTLP receiver that verifies one chat request produces correlated gateway and vLLM spans without prompt/completion content.

Acceptance criteria

• Tracing is disabled by default and can be enabled through documented chart/model configuration.
• Gateway and vLLM spans share trace/session correlation where upstream context is present.
• Streaming chat completions produce terminal spans with status, latency, model, and token usage.
• Prompt and completion content are absent under the default privacy configuration.
• Sampling and detailed-trace settings are independently configurable.
• Generated CRDs, Helm examples, operator reconciliation, and E2E coverage include the new fields.
• The design works without a LangChain dependency.

Dependencies

• Parent tracking issue: nebari-dev/nebari-nexus-pack#55
• Internal deployment: openteams-ai/nexus.internal.openteams.app#26

Official references:

• Envoy AI Gateway GenAI distributed tracing
• vLLM OpenTelemetry setup
• vLLM observability configuration