Skip to content

Commit 5964019

Browse files
scottgerringscottgerring
committed
chore: Add OTLP docs and general arch
1 parent 34d6d50 commit 5964019

File tree

2 files changed

+217
-0
lines changed

2 files changed

+217
-0
lines changed

‎docs/design/architecture.md

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# OpenTelemetry Rust – Architecture Overview
2+
3+
## 1 Purpose
4+
This document provides a stable, high-level description of how the OpenTelemetry Rust implementation is put together. Detailed per-signal design notes live in their own files; this overview ties the pieces together.
5+
6+
> Reference: the [OpenTelemetry Specification](https://opentelemetry.io/docs/specs/otel/) and its [GitHub repository](https://github.com/open-telemetry/opentelemetry-specification/)
7+
8+
## 2 Layering
9+
```mermaid
10+
graph TD
11+
A[Application code] --> B[opentelemetry API]
12+
B --> C[opentelemetry-sdk]
13+
C --> D[Processors / Readers]
14+
D --> E[Exporters]
15+
E -->|"protocols (OTLP, Zipkin, Prometheus, stdout)"| F[Back-end / Collector]
16+
```
17+
Key points:
18+
1. The **API** crate is a lightweight facade that library and application code instrument against.
19+
2. The **SDK** crate supplies concrete implementations, batching, aggregation, and lifecycle management.
20+
• **Processors / readers** live inside the SDK and adapt buffering and temporality.
21+
3. **Exporters** translate OTel data into wire formats (OTLP, Prometheus, etc.) and handle transport.
22+
23+
## 3 Cross-cutting Components
24+
* **Resource** – a representation of the entity producing telemetry - see the [resource spec](https://opentelemetry.io/docs/specs/otel/resource/).
25+
* **Attributes** - a key-value pair used to annotate telemetry data - see the [common spec](https://opentelemetry.io/docs/specs/otel/common/#attributes).
26+
* **Context & Propagation** – context management and carrier injection; see the [context propagation spec](https://opentelemetry.io/docs/specs/otel/context/).
27+
* **Runtime model** — public APIs are agnostic to execution model (synchronous or asynchronous); implementations may support optional async exporters (e.g., Tokio) or blocking I/O, depending on configuration. See the [SDK design guidelines](https://opentelemetry.io/docs/specs/otel/).
28+
* **Error taxonomy** – see [ADR 001 Error Handling](../adr/001_error_handling.md).
29+
30+
## Detailed Design
31+
32+
### Signals
33+
34+
* **Traces** – [design doc](./traces.md)
35+
* **Metrics** – [design doc](./metrics.md)
36+
* **Logs** – [design doc](./logs.md)
37+
38+
### Exporters
39+
40+
* **OTLP** – [design doc](./otlp.md) — the OTLP exporter.
41+
42+
## 5 Extensibility Hooks
43+
| Layer | Customisation points |
44+
|-------|---------------------|
45+
| SDK | Provide complete alternative SDK |
46+
| SDK | Plug-in **samplers**, **metric readers**, **log processors**, and **providers** |
47+
| Exporters | Support additional wire-protocols |
48+
| Processors | Provide alternative `LogProcessor` or `SpanProcessor` |

‎docs/design/otlp.md

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
# OpenTelemetry Rust – OTLP Architecture
2+
3+
This document describes how the **OTLP exporters** are organised and how they integrate with the wider OpenTelemetry Rust SDK.
4+
5+
> Reference the [OTLP specification](https://opentelemetry.io/docs/specs/otlp/) and [OTLP Protocol Exporter specification](https://opentelemetry.io/docs/specs/otel/protocol/exporter/).
6+
7+
## Overview
8+
OpenTelemetry Protocol (OTLP) is the vendor-agnostic wire format used by OpenTelemetry to transport telemetry signals (traces, metrics, logs) from instrumented code to a backend (OTel Collector or compatible vendor). It supports:
9+
10+
* **Encodings**
11+
* *Protobuf* – canonical format
12+
* *JSON* (optional)
13+
* **Transports**
14+
* *gRPC* (default port `4317`)
15+
* *HTTP/1.1*, *HTTP/2* (default port `4318`, binary protobuf or JSON payloads)
16+
* **Signals**
17+
* Traces, Metrics, Logs – transported independently but share common envelope (`Resource`, `Scope` etc.)
18+
19+
## 2. Overall Structure of `opentelemetry-rust` and OTLP
20+
```mermaid
21+
graph TD
22+
A[Application code] --> B[opentelemetry-api]
23+
B --> C[opentelemetry-sdk processors]
24+
C --> D[opentelemetry-otlp exporters]
25+
D -->|gRPC / HTTP| E[OTel Collector / Backend]
26+
%% Auxiliary exporters
27+
C --> F[custom exporters]
28+
```
29+
30+
Important details:
31+
32+
* `opentelemetry-sdk` owns batching, aggregation, lifecycle; **exporters handle serialization, transport, and may implement retry policy**.
33+
* Each signal (trace/metric/log) has its own exporter type but they share common builder + transport modules.
34+
35+
---
36+
37+
## 3. Crate Layout
38+
```text
39+
src/
40+
lib.rs # Re-exports + feature flags + protocol enum
41+
span.rs | metric.rs | logs.rs # Signal-specific builders/exporters
42+
exporter/
43+
├── mod.rs # Common builder traits, env var parsing, compression
44+
├── http/ # Reqwest blocking/async clients, body encoder
45+
└── tonic/ # Tonic clients, TLS/compression helpers
46+
```
47+
### 3.1 Common Interfaces
48+
* **`HasExportConfig` / `WithExportConfig`** – traits mixed into builders to expose shared config (`endpoint`, `timeout`, `protocol`).
49+
* **Builder marker types** (`NoExporterBuilderSet`, `HttpExporterBuilderSet`, `TonicExporterBuilderSet`) enforce at compile time that exactly *one* transport is chosen.
50+
* **`SupportedTransportClient` enum** – run-time dispatch inside the exporter when sending.
51+
52+
---
53+
54+
## 4. Feature-Flag Matrix
55+
| Cargo feature | Purpose | Conditional modules |
56+
|---------------|---------|---------------------|
57+
| `trace` / `metrics` / `logs` | Enable signal exporters | `span.rs`, `metric.rs`, `logs.rs` |
58+
| `grpc-tonic` | Use `tonic` gRPC transport | `exporter/tonic` |
59+
| `http-proto` *(default)* | HTTP + protobuf body | `exporter/http` |
60+
| `http-json` | HTTP + JSON body | `exporter/http` |
61+
| `gzip-tonic` `zstd-tonic` | gRPC message compression | `exporter/tonic` |
62+
| `reqwest-client` / `reqwest-blocking-client` *(default)* | Choose async vs blocking Reqwest HTTP client |
63+
| `hyper-client` | Use Hyper HTTP transport *(requires both Reqwest features disabled)* |
64+
| TLS helpers (`tls-roots`, `tls-webpki-roots`, `reqwest-rustls`) | Supply trust roots for TLS |
65+
66+
Because **only one transport is valid per exporter**, `protocol` is currently a *hint* – unsupported values are ignored by the concrete builder.
67+
68+
### 4.1 Transport-specific Client Options
69+
70+
Exporters must function both in binaries that start no async runtime and in services already running Tokio. Consequently the crate provides **blocking** and **non-blocking** HTTP clients behind mutually-exclusive feature-flags. The blocking variant is the default to ensure out-of-the-box operability; applications that already depend on Tokio can opt into an async client to avoid spawning extra threads.
71+
72+
| Implementation | Feature flag | Blocking? | Underlying crate | Transports | Typical use case |
73+
|---------------|--------------|-----------|------------------|-----------|------------------|
74+
| Reqwest (blocking) **(default)** | `reqwest-blocking-client` | Yes | `reqwest::blocking` | HTTP/1.1 → auto-upgrades to HTTP/2 over TLS | CLI tools, synchronous binaries, or hybrid apps where introducing Tokio is undesirable. A helper thread is spawned once at builder time.
75+
| Reqwest (async) | `reqwest-client` | No (Tokio) | `reqwest` | HTTP/1.1 → auto-upgrades to HTTP/2 over TLS | Services that already run a Tokio runtime and prefer fully non-blocking exports.
76+
| Hyper | `hyper-client` | No (Tokio) | `hyper` | HTTP/1.1 → auto-upgrades to HTTP/2 over TLS | Lean dependency footprint when both Reqwest features are disabled.
77+
| Tonic (gRPC) | `grpc-tonic` | No (Tokio) | `tonic` | HTTP/2 (gRPC) | Environments standardising on gRPC or collectors exposing only port 4317.
78+
| Custom | — | Depends | user-supplied | depends on implementation | Embedding a bespoke `HttpClient` via `.with_http_client()`.
79+
80+
---
81+
82+
## 5. Runtime Flow (Trace Export)
83+
```mermaid
84+
sequenceDiagram
85+
participant App as Application code
86+
participant SDK as opentelemetry-sdk
87+
participant Exp as SpanExporter
88+
participant Tx as Transport Client (HTTP/Reqwest or gRPC/Tonic)
89+
participant Collector as OTLP Collector
90+
App->>SDK: span.start / span.end
91+
SDK-->>SDK: BatchSpanProcessor collects SpanData
92+
SDK->>Exp: export(batch)
93+
Exp->>Tx: serialize + send
94+
Tx--)Collector: OTLP request
95+
note right of Collector: 2xx = success,<br/>non-2xx → error handling
96+
Tx--)Exp: Future resolves
97+
Exp--)SDK: OTelSdkResult
98+
```
99+
Notes:
100+
101+
* Serialization happens **inside the transport client** module to keep exporters thin.
102+
* `export` is `async` for `tonic` / `reqwest` clients, surfaced via `SpanExporter` implementing `opentelemetry_sdk::trace::SpanExporter`.
103+
* Resource attributes are injected once per exporter via `set_resource()` before first export.
104+
105+
---
106+
107+
## 6. Configuration & Environment Variable Resolution
108+
`exporter::mod.rs` implements helper fns:
109+
* `default_protocol()` – chosen from compile-time defaults.
110+
* `resolve_timeout()` – precedence: signal-specific env → generic env → builder → default 10 s.
111+
* `parse_header_string()` – parses comma-separated `k=v` pairs with URL-decoding.
112+
113+
Signal builders read **signal-specific env vars** (e.g. `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`) _before_ generic ones, matching the spec.
114+
115+
---
116+
117+
## 7. Error Handling Strategy
118+
`ExporterBuildError` is a non-exhaustive enum covering:
119+
* Builder-time validation (URI parse, missing client)
120+
* Feature gating errors (compression requested w/o feature)
121+
* Runtime errors are wrapped in `OTelSdkError` from the SDK.
122+
123+
Builder fails fast, runtime exporter surfaces errors through `export()` futures so processors (or Retry logic) can decide whether to back-off, drop, or escalate.
124+
125+
---
126+
127+
## 8. Extension & Customisation Points
128+
1. **Custom headers / metadata** – `.with_metadata(map)` (gRPC) or `.with_headers()` (HTTP).
129+
2. **Compression** – `.with_compression(Compression::Gzip | Compression::Zstd)` behind feature-flags.
130+
3. **TLS** – via `TonicConfig` or `HttpConfig`; root-store helper features embed common CA bundles.
131+
4. **Alternate HTTP client** – inject any `HttpClient` implementation via `.with_http_client()`.
132+
5. **Protocol JSON** – switch to JSON payloads at build-time with the `http-json` feature.
133+
134+
---
135+
136+
## 9. Interactions with Other Exporters
137+
* **Prometheus exporter** (pull-based) bypasses OTLP entirely.
138+
* **Jaeger / Zipkin** exporters are alternative push paths; users typically pick *one* per signal.
139+
* **stdout exporter** may be enabled alongside OTLP in development.
140+
141+
Example mixed configuration:
142+
```rust
143+
let otlp = SpanExporter::builder().with_tonic().build()?;
144+
let jaeger = opentelemetry_jaeger::new_agent_pipeline().install_simple()?;
145+
let provider = SdkTracerProvider::builder()
146+
.with_batch_exporter(otlp)
147+
.with_simple_exporter(jaeger)
148+
.build();
149+
```
150+
151+
---
152+
153+
## 10. Key Architectural Decisions
154+
| Decision | Rationale |
155+
|----------|-----------|
156+
| *Builder pattern with marker types* | Compile-time guarantee that exactly one transport is chosen. |
157+
| *Transport-specific modules* | Keep heavy deps (`tonic`, `reqwest`) behind feature-flags to minimise compile times. |
158+
| *Separate crate* | Avoids pulling heavy deps into projects that don’t need OTLP. |
159+
| *Non-exhaustive error enums* | Allows adding new failure modes without breaking semver. |
160+
161+
---
162+
163+
### Source References
164+
* Builder traits – [`exporter/mod.rs`](../../opentelemetry-otlp/src/exporter/mod.rs)
165+
* HTTP transport – [`exporter/http/mod.rs`](../../opentelemetry-otlp/src/exporter/http/mod.rs)
166+
* gRPC transport – [`exporter/tonic/mod.rs`](../../opentelemetry-otlp/src/exporter/tonic/mod.rs)
167+
* Span exporter – [`span.rs`](../../opentelemetry-otlp/src/span.rs)
168+
* Metric exporter – [`metric.rs`](../../opentelemetry-otlp/src/metric.rs)
169+
* Logs exporter – [`logs.rs`](../../opentelemetry-otlp/src/logs.rs)

0 commit comments

Comments
 (0)