Skip to content

Commit 41f088d

Browse files
scottgerringscottgerring
committed
chore: Add OTLP docs and general arch
1 parent 34d6d50 commit 41f088d

File tree

2 files changed

+203
-0
lines changed

2 files changed

+203
-0
lines changed

‎docs/design/architecture.md

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# OpenTelemetry Rust – Architecture Overview
2+
3+
> Status:Development
4+
> Last Updated: 2025-07-16
5+
6+
## 1 Purpose
7+
This document provides a high-level description of how the OpenTelemetry Rust implementation is put together. Detailed per-signal design notes live in their own files; this overview ties the pieces together.
8+
9+
> Reference: the [OpenTelemetry Specification](https://opentelemetry.io/docs/specs/otel/) and its [GitHub repository](https://github.com/open-telemetry/opentelemetry-specification/)
10+
11+
## 2 Layering
12+
```mermaid
13+
graph TD
14+
A[Application code] --> B[opentelemetry API]
15+
B --> C[opentelemetry-sdk]
16+
C --> D[Processors / Readers]
17+
D --> E[Exporters]
18+
E -->|"protocols (OTLP, Zipkin, Prometheus, stdout)"| F[Back-end / Collector]
19+
```
20+
Key points:
21+
1. The **API** crate is a lightweight facade that library and application code instrument against.
22+
2. The **SDK** crate supplies concrete implementations, batching, aggregation, and lifecycle management.
23+
• **Processors / readers** live inside the SDK and adapt buffering and temporality.
24+
3. **Exporters** translate OTel data into wire formats (OTLP, Prometheus, etc.) and handle transport.
25+
26+
## 3 Cross-cutting Components
27+
* **Resource** – a representation of the entity producing telemetry - see the [resource spec](https://opentelemetry.io/docs/specs/otel/resource/).
28+
* **Attributes** - a key-value pair used to annotate telemetry data - see the [common spec](https://opentelemetry.io/docs/specs/otel/common/#attributes).
29+
* **Context & Propagation** – context management and carrier injection; see the [context propagation spec](https://opentelemetry.io/docs/specs/otel/context/).
30+
* **Runtime model** — public APIs are agnostic to execution model (synchronous or asynchronous); implementations may support optional async exporters (e.g., Tokio) or blocking I/O, depending on configuration.
31+
* **Error taxonomy** – see [ADR 001 Error Handling](../adr/001_error_handling.md).
32+
33+
## Detailed Design
34+
35+
### Signals
36+
37+
* **Traces** – [design doc](./traces.md)
38+
* **Metrics** – [design doc](./metrics.md)
39+
* **Logs** – [design doc](./logs.md)
40+
41+
### Exporters
42+
43+
* **OTLP** – [design doc](./otlp.md) — the OTLP exporter.
44+
45+
## 5 Extensibility Hooks
46+
| Layer | Customisation points |
47+
|-------|---------------------|
48+
| SDK | Provide complete alternative SDK |
49+
| SDK | Plug-in **samplers**, **metric readers**, **log processors**, and **providers** |
50+
| Exporters | Support additional wire-protocols |

‎docs/design/otlp.md

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
# OpenTelemetry Rust – OTLP Architecture
2+
3+
> Status:Development
4+
> Last Updated: 2025-07-16
5+
6+
This document describes how the **OTLP exporters** are organised and how they integrate with the wider OpenTelemetry Rust SDK.
7+
8+
> Reference the [OTLP specification](https://opentelemetry.io/docs/specs/otlp/) and [OTLP Protocol Exporter specification](https://opentelemetry.io/docs/specs/otel/protocol/exporter/).
9+
10+
## Overview
11+
OpenTelemetry Protocol (OTLP) is the vendor-agnostic wire format used by OpenTelemetry to transport telemetry signals (traces, metrics, logs) from instrumented code to a backend (OTel Collector or compatible vendor). It supports:
12+
13+
* **Encodings**
14+
* *Protobuf* – canonical format
15+
* *JSON* (optional)
16+
* **Transports**
17+
* *gRPC* (default port `4317`)
18+
* *HTTP/1.1*, *HTTP/2* (default port `4318`, binary protobuf or JSON payloads)
19+
* **Signals**
20+
* Traces, Metrics, Logs – transported independently but share common envelope types (`Resource`, `Scope` etc.)
21+
22+
## 2. Overall Structure of `opentelemetry-rust` and OTLP
23+
```mermaid
24+
graph TD
25+
A[Application code] --> B[opentelemetry-api]
26+
B --> C[opentelemetry-sdk processors]
27+
C --> D[opentelemetry-otlp exporters]
28+
D -->|gRPC / HTTP| E[OTel Collector / Backend]
29+
%% Auxiliary exporters
30+
C --> F[custom exporters]
31+
```
32+
33+
Important details:
34+
35+
* `opentelemetry-sdk` owns batching, aggregation, lifecycle; **exporters handle serialization, transport, and may implement retry policy**.
36+
* Each signal (trace/metric/log) has its own exporter type but they share common builder + transport modules.
37+
38+
---
39+
40+
## 3. Crate Layout
41+
```text
42+
src/
43+
lib.rs # Re-exports + feature flags + protocol enum
44+
span.rs | metric.rs | logs.rs # Signal-specific builders/exporters
45+
exporter/
46+
├── mod.rs # Common builder traits, env var parsing, compression
47+
├── http/ # Reqwest blocking/async clients, body encoder
48+
└── tonic/ # Tonic client, TLS/compression helpers
49+
```
50+
### 3.1 Common Interfaces
51+
* **`HasExportConfig` / `WithExportConfig`** – traits mixed into builders to expose shared config (`endpoint`, `timeout`, `protocol`).
52+
* **Builder marker types** (`NoExporterBuilderSet`, `HttpExporterBuilderSet`, `TonicExporterBuilderSet`) enforce at compile time that exactly *one* transport is chosen.
53+
* **`SupportedTransportClient` enum** – run-time dispatch inside the exporter when sending.
54+
55+
---
56+
57+
## 4. Feature-Flag Matrix
58+
| Cargo feature | Purpose | Conditional modules |
59+
|---------------|---------|---------------------|
60+
| `trace` / `metrics` / `logs` | Enable signal exporters | `span.rs`, `metric.rs`, `logs.rs` |
61+
| `grpc-tonic` | Use `tonic` gRPC transport | `exporter/tonic` |
62+
| `http-proto` *(default)* | HTTP + protobuf body | `exporter/http` |
63+
| `http-json` | HTTP + JSON body | `exporter/http` |
64+
| `gzip-tonic` `zstd-tonic` | gRPC message compression | `exporter/tonic` |
65+
| `reqwest-client` / `reqwest-blocking-client` *(default)* | Choose async vs blocking Reqwest HTTP client |
66+
| `hyper-client` | Use Hyper HTTP transport *(requires both Reqwest features disabled)* |
67+
| TLS helpers (`tls-roots`, `tls-webpki-roots`, `reqwest-rustls`) | Supply trust roots for TLS |
68+
69+
Because **only one transport is valid per exporter**, `protocol` is currently a *hint* – unsupported values are ignored by the concrete builder.
70+
71+
### 4.1 Transport-specific Client Options
72+
73+
Exporters must function both in binaries that start no async runtime and in services already running async runtimes such as Tokio. Consequently the crate provides **blocking** and **non-blocking** HTTP clients behind mutually-exclusive feature-flags. The blocking variant is the default to ensure out-of-the-box operability; applications that already depend on Tokio can opt into an async client to avoid spawning extra threads.
74+
75+
| Implementation | Feature flag | Blocking? | Underlying crate | Transports | Typical use case |
76+
|---------------|--------------|-----------|------------------|-----------|------------------|
77+
| Reqwest (synchronous) **(default)** | `reqwest-blocking-client` | Yes | `reqwest::blocking` | HTTP/1.1 → auto-upgrades to HTTP/2 over TLS | CLI tools, synchronous binaries, or hybrid apps where introducing Tokio is undesirable. A helper thread is spawned once at builder time.
78+
| Reqwest (async) | `reqwest-client` | No (Tokio) | `reqwest` | HTTP/1.1 → auto-upgrades to HTTP/2 over TLS | Services that already run a Tokio runtime and prefer fully non-blocking exports.
79+
| Hyper | `hyper-client` | No (Tokio) | `hyper` | HTTP/1.1 → auto-upgrades to HTTP/2 over TLS | Lean dependency footprint when both Reqwest features are disabled.
80+
| Tonic (async gRPC) | `grpc-tonic` | No (Tokio) | `tonic` | HTTP/2 (gRPC) | A desire to use gRPC exporter.
81+
| Custom | — | Depends | user-supplied | depends on implementation | Embedding a bespoke `HttpClient` via `.with_http_client()`.
82+
83+
---
84+
85+
## 5. Runtime Flow (Trace Export)
86+
```mermaid
87+
sequenceDiagram
88+
participant App as Application code
89+
participant SDK as opentelemetry-sdk
90+
participant Exp as SpanExporter
91+
participant Tx as Transport Client (HTTP/Reqwest or gRPC/Tonic)
92+
participant Collector as OTLP Collector
93+
App->>SDK: span.start / span.end
94+
SDK-->>SDK: BatchSpanProcessor collects SpanData
95+
SDK->>Exp: export(batch)
96+
Exp->>Tx: serialize + send
97+
Tx--)Collector: OTLP request
98+
note right of Collector: 2xx = success,<br/>non-2xx → error handling
99+
Tx--)Exp: Future resolves
100+
Exp--)SDK: OTelSdkResult
101+
```
102+
Notes:
103+
104+
* Serialization happens **inside the transport client** module to keep exporters thin.
105+
* `export` methods return futures, surfaced via signal exporters implementing their respective exporter traits (`SpanExporter`, `PushMetricExporter`, `LogExporter`).
106+
* Resource attributes are injected once per exporter via `set_resource()` before first export.
107+
108+
---
109+
110+
## 6. Configuration & Environment Variable Resolution
111+
`exporter::mod.rs` implements helper fns:
112+
* `default_protocol()` – chosen from compile-time defaults.
113+
* `resolve_timeout()` – precedence: signal-specific env → generic env → builder → default 10 s.
114+
* `parse_header_string()` – parses comma-separated `k=v` pairs with URL-decoding.
115+
116+
Signal builders read **signal-specific env vars** (e.g. `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`) _before_ generic ones, matching the spec.
117+
118+
---
119+
120+
## 7. Error Handling Strategy
121+
`ExporterBuildError` is a non-exhaustive enum covering:
122+
* Builder-time validation (URI parse, missing client)
123+
* Feature gating errors (compression requested w/o feature)
124+
* Runtime errors are wrapped in `OTelSdkError` from the SDK.
125+
126+
Builder fails fast, runtime exporter surfaces errors through `export()` futures so processors (or Retry logic) can decide whether to back-off, drop, or escalate.
127+
128+
---
129+
130+
## 8. Extension & Customisation Points
131+
1. **Custom headers / metadata** – `.with_metadata(map)` (gRPC) or `.with_headers()` (HTTP).
132+
2. **Compression** – `.with_compression(Compression::Gzip | Compression::Zstd)` behind feature-flags.
133+
3. **TLS** – via `TonicConfig` or `HttpConfig`; root-store helper features embed common CA bundles.
134+
4. **Alternate HTTP client** – inject any `HttpClient` implementation via `.with_http_client()`.
135+
5. **Protocol JSON** – switch to JSON payloads at build-time with the `http-json` feature.
136+
137+
---
138+
139+
## 9. Key Architectural Decisions
140+
| Decision | Rationale |
141+
|----------|-----------|
142+
| *Builder pattern with marker types* | Compile-time guarantee that exactly one transport is chosen. |
143+
| *Transport-specific modules* | Keep heavy deps (`tonic`, `reqwest`) behind feature-flags to minimise compile times. |
144+
145+
---
146+
147+
### Source References
148+
* Builder traits – [`exporter/mod.rs`](../../opentelemetry-otlp/src/exporter/mod.rs)
149+
* HTTP transport – [`exporter/http/mod.rs`](../../opentelemetry-otlp/src/exporter/http/mod.rs)
150+
* gRPC transport – [`exporter/tonic/mod.rs`](../../opentelemetry-otlp/src/exporter/tonic/mod.rs)
151+
* Span exporter – [`span.rs`](../../opentelemetry-otlp/src/span.rs)
152+
* Metric exporter – [`metric.rs`](../../opentelemetry-otlp/src/metric.rs)
153+
* Logs exporter – [`logs.rs`](../../opentelemetry-otlp/src/logs.rs)

0 commit comments

Comments
 (0)