OpenTelemetry APM Playground

E2E example of application performance monitoring (APM) and distributed tracing using OpenTelemetry across Python services. Trace is further used in writing integration tests.

What this project does

• Generates traces from a Python Flask "Banking API" that exposes user and transaction endpoints and exports spans via OTLP/HTTP.
• Captures rich HTTP context in spans from the Python API, including request and response bodies, to make traces more useful for debugging and test generation.
• Receives and processes traces with a minimal Python "processing service" that accepts OTLP over HTTP (JSON or Protobuf), decodes them to JSON, and persists span data to MongoDB.
• Persists route intelligence by de‑duplicating and storing a unique identifier per route/method, enabling analysis and potential replay of production traffic from captured span attributes.

Components at a glance

• python/user-service/
  • Flask‑RESTx API for a simple banking domain (users, credit/debit/transfer).
  • OpenTelemetry Python SDK with OTLP/HTTP exporter and Flask auto‑instrumentation.
  • Adds custom span attributes for http.request.body and http.response.body while avoiding noise from Swagger assets.
• python/processing-service/
  • Minimal OTLP HTTP trace receiver (/v1/traces) that accepts application/json and application/x-protobuf.
  • Decodes ExportTraceServiceRequest to JSON, extracts HTTP attributes, and saves spans to MongoDB with a unique route key (method + route).
  • Utility helpers to interpret span attributes and (optionally) reconstruct outbound HTTP calls from captured data.
• python/notebooks/
  • Notebook(s) intended for exploring captured traces and generating tests or examples from stored span data.

How it fits together (data flow)

1. The Python Banking API handles HTTP requests and emits OpenTelemetry spans.
2. The Banking API exports spans to the local processing service using OTLP over HTTP.
3. The processing service decodes incoming traces, normalizes them to JSON, and stores select fields and the full span document in MongoDB.
4. Stored spans provide a lightweight knowledge base of real routes and payloads that can be analyzed, visualized elsewhere, or used to drive synthetic tests.

Why this is useful

• Demonstrates practical OpenTelemetry usage in Python with the OTLP exporter.
• Shows how to enrich spans with HTTP bodies for deeper observability when appropriate.
• Provides a simple, hackable trace receiver for learning, prototyping, and building trace‑driven tooling (e.g., test generation or traffic replay).

Technologies

• OpenTelemetry SDK (Python), OTLP/HTTP
• Flask, Flask‑RESTx
• MongoDB for span persistence