Skip to content

Commit f1a9b4a

Browse files
PavelPashovPavelPashov
authored
feat: add OpenTelemetry metrics instrumentation (redis#3110)
* wip: add OpenTelemetry metrics instrumentation * add noop metrics * fix: check if metrics are initilized in sendCommand * fix(otel): optimize metrics tracking by eliminating promise chaining overhead * refactor(otel): organize metrics into specialized metric groups * fix(otel): revert metrics to be tracked in the sendCommand method * perf(metrics): optimize command metrics with factory pattern and inline attributes * feat: Add new OTEL_ATTRIBUTES constants * feat: Add client-side-caching metric group with 4 new metric names * feat: Create error categorization stub function * feat: Add pool name formatting utility function * feat: Refactor recordConnectionCreateTime to use closure pattern * feat: Rename redis.client.errors.handled to redis.client.errors * feat: Update IOTelResiliencyMetrics interface with internal flag and error attributes * feat: Add redis.client.connection.notification attribute to maintenance metrics * feat: Add db.response.status_code attribute extraction for Redis errors * feat: Wire redis.client.connection.closed metric with close reason attribute * feat: Add db.client.connection.wait_time method with closure pattern * feat: Add db.client.connection.use_time method with closure pattern * feat: Integrate wait_time and use_time metrics into connection pool * feat: Add CSC metric instruments to registerInstruments * feat: Create IOTelClientSideCacheMetrics interface and implementation class * feat: Wire redis.client.csc.requests metric to cache hit/miss detection * feat: Wire redis.client.csc.items metric to track cache size changes * feat: Wire redis.client.csc.evictions metric with eviction reason * feat: Wire redis.client.csc.network_saved metric to estimate bytes saved on cache hit * refactor: Add count parameter to recordCacheEviction to batch eviction recording * fix: typo * feat: Implement redis.client.pubsub.messages metric for pub/sub publish and receive paths * feat: Implement redis.client.stream.produce.messages metric for stream producers * refactor: move pub/sub and stream metrics recording to command wrapper * refactor: align metric groups with instrumentation spec * feat: add error classification helper and enrich metrics attributes * refactor: replace command wrapper with onSuccess hook for metrics * feat(client): add client identity tracking for OpenTelemetry metrics * feat(otel): convert metrics to observable gauges with client registry * feat(otel): refine metrics coverage and remove connection use_time instrument * refactor(otel): resolve metric attributes via clientId registry lookup * refactor(otel): propagate clientId through runtime metrics paths * test(otel): expand metrics coverage and add test utilities * test(otel): fix flaky test * feat(otel): align metric attributes/config and expand observability coverage * test(otel): add maintenance metrics e2e scenario with standalone FI config * refactor(otel): use instrumentation scope name and stop injecting resource attrs * fix(opentelemetry): rename stream bucket config * docs(opentelemetry): add metrics docs/examples * fix(otel): scope redirection error dedupe to cluster retry path * fix(opentelemetry): normalize db.namespace and server.port to strings in command and CSC metrics * fix(otel): disable recordNetworkBytesSaved for CSC * refactor(otel): remove command.onSuccess * feat(otel): convert connection count metric to UpDownCounter (redis#6) * fix: keep registry identity in sync after _setIdentity
1 parent e75e314 commit f1a9b4a

41 files changed

Lines changed: 6092 additions & 180 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎README.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -326,6 +326,33 @@ See the [Programmability overview](https://github.com/redis/node-redis/blob/mast
326326

327327
Check out the [Clustering Guide](https://github.com/redis/node-redis/blob/master/docs/clustering.md) when using Node Redis to connect to a Redis Cluster.
328328

329+
### OpenTelemetry
330+
331+
#### OpenTelemetry Metrics Instrumentation
332+
333+
```typescript
334+
import { createClient, OpenTelemetry } from "redis";
335+
336+
OpenTelemetry.init({
337+
metrics: {
338+
enabled: true
339+
}
340+
});
341+
342+
const client = createClient()
343+
344+
await client.connect();
345+
// ... use the client as usual
346+
```
347+
348+
**Important:** Initializing `OpenTelemetry` only enables node-redis metrics instrumentation and requires both `@opentelemetry/api` and an OpenTelemetry SDK configured in your application.
349+
350+
**Important:** Initialize `OpenTelemetry` before creating Redis clients.
351+
For SDK/provider/exporter setup, verification, and advanced configuration, see:
352+
353+
- [OpenTelemetry Metrics docs](./docs/otel-metrics.md)
354+
- [OpenTelemetry Metrics example](./examples/otel-metrics.js)
355+
329356
### Events
330357

331358
The Node Redis client class is an Nodejs EventEmitter and it emits an event each time the network status changes:

‎docs/otel-metrics.md‎

Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
# OpenTelemetry Metrics
2+
3+
## Get started
4+
5+
### Step 1. Install node-redis dependencies
6+
7+
```bash
8+
npm install redis @opentelemetry/api
9+
```
10+
11+
`@opentelemetry/api` is required at runtime for `OpenTelemetry.init(...)`.
12+
13+
### Step 2. Install OpenTelemetry SDK packages
14+
15+
```bash
16+
npm install @opentelemetry/sdk-metrics
17+
```
18+
19+
Alternative (Node SDK):
20+
21+
```bash
22+
npm install @opentelemetry/sdk-node @opentelemetry/sdk-metrics
23+
```
24+
25+
If you export to OTLP or another backend, install the matching OpenTelemetry exporter package.
26+
27+
For more information, see the [OpenTelemetry Metrics documentation](https://opentelemetry.io/docs/instrumentation/js/exporters/#metrics).
28+
29+
### Step 3. Register OpenTelemetry
30+
31+
Option A: Use `@opentelemetry/sdk-metrics` directly
32+
33+
```typescript
34+
import { metrics } from "@opentelemetry/api";
35+
import {
36+
ConsoleMetricExporter,
37+
MeterProvider,
38+
PeriodicExportingMetricReader,
39+
} from "@opentelemetry/sdk-metrics";
40+
41+
const meterProvider = new MeterProvider({
42+
readers: [
43+
new PeriodicExportingMetricReader({
44+
exporter: new ConsoleMetricExporter(),
45+
exportIntervalMillis: 1000,
46+
}),
47+
],
48+
});
49+
50+
metrics.setGlobalMeterProvider(meterProvider);
51+
```
52+
53+
Option B: Use `@opentelemetry/sdk-node`
54+
55+
```typescript
56+
import { NodeSDK } from "@opentelemetry/sdk-node";
57+
import {
58+
ConsoleMetricExporter,
59+
PeriodicExportingMetricReader,
60+
} from "@opentelemetry/sdk-metrics";
61+
62+
const sdk = new NodeSDK({
63+
metricReader: new PeriodicExportingMetricReader({
64+
exporter: new ConsoleMetricExporter(),
65+
exportIntervalMillis: 1000,
66+
}),
67+
});
68+
69+
await sdk.start();
70+
```
71+
72+
### Step 4. Initialize node-redis instrumentation before creating clients
73+
74+
```typescript
75+
import { createClient, OpenTelemetry } from "redis";
76+
77+
OpenTelemetry.init({
78+
metrics: {
79+
enabled: true,
80+
},
81+
});
82+
83+
const client = createClient();
84+
await client.connect();
85+
```
86+
87+
## Examples
88+
89+
### Minimal Example
90+
91+
```typescript
92+
import { OpenTelemetry } from "redis";
93+
94+
OpenTelemetry.init({
95+
metrics: {
96+
enabled: true,
97+
},
98+
});
99+
```
100+
101+
### Full Example
102+
103+
```typescript
104+
import { OpenTelemetry } from "redis";
105+
106+
OpenTelemetry.init({
107+
metrics: {
108+
enabled: true,
109+
meterProvider: customMeterProvider,
110+
enabledMetricGroups: ["command", "pubsub", "streaming", "resiliency"],
111+
includeCommands: ["GET", "HSET", "XREADGROUP", "PUBLISH"],
112+
excludeCommands: ["SET"],
113+
hidePubSubChannelNames: true,
114+
hideStreamNames: false,
115+
bucketsOperationDuration: [0.001, 0.01, 0.1, 1],
116+
bucketsStreamProcessingDuration: [0.01, 0.1, 1, 5],
117+
},
118+
});
119+
```
120+
121+
## Configuration
122+
123+
### ObservabilityConfig
124+
125+
| Property | Default | Description |
126+
| -------- | ------- | ----------- |
127+
| metrics | | OpenTelemetry metrics configuration for node-redis. |
128+
129+
### MetricConfig
130+
131+
| Property | Default | Description |
132+
| -------- | ------- | ----------- |
133+
| enabled | **false** | Enables metric collection. |
134+
| meterProvider | | Uses this provider instead of the global provider from @opentelemetry/api. |
135+
| includeCommands | **[]** | Case-insensitive allow-list for command metrics. |
136+
| excludeCommands | **[]** | Case-insensitive deny-list for command metrics. If both include and exclude match, exclude wins. |
137+
| enabledMetricGroups | **['connection-basic', 'resiliency']** | Metric groups to enable. Supported groups: command, connection-basic, connection-advanced, resiliency, pubsub, streaming, client-side-caching. |
138+
| hidePubSubChannelNames | **false** | If true, omits redis.client.pubsub.channel to reduce cardinality. |
139+
| hideStreamNames | **false** | If true, omits redis.client.stream.name to reduce cardinality. |
140+
| bucketsOperationDuration | **[0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10]** | Histogram bucket boundaries for db.client.operation.duration (seconds). |
141+
| bucketsConnectionCreateTime | **[0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10]** | Histogram bucket boundaries for db.client.connection.create_time (seconds). |
142+
| bucketsConnectionWaitTime | **[0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10]** | Histogram bucket boundaries for db.client.connection.wait_time (seconds). |
143+
| bucketsStreamProcessingDuration | **[0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10]** | Histogram bucket boundaries for redis.client.stream.lag (seconds). |
144+
145+
## Metric groups and metrics
146+
147+
| Metric Group | Metric Name |
148+
| ------------ | ----------- |
149+
| command | db.client.operation.duration |
150+
| connection-basic | db.client.connection.count |
151+
| connection-basic | db.client.connection.create_time |
152+
| connection-basic | redis.client.connection.relaxed_timeout |
153+
| connection-basic | redis.client.connection.handoff |
154+
| connection-advanced | db.client.connection.wait_time |
155+
| connection-advanced | redis.client.connection.closed |
156+
| resiliency | redis.client.errors |
157+
| resiliency | redis.client.maintenance.notifications |
158+
| pubsub | redis.client.pubsub.messages |
159+
| streaming | redis.client.stream.lag |
160+
| client-side-caching | redis.client.csc.requests |
161+
| client-side-caching | redis.client.csc.items |
162+
| client-side-caching | redis.client.csc.evictions |
163+
| client-side-caching | redis.client.csc.network_saved |
164+
165+
## Notes
166+
167+
- `OpenTelemetry` is a singleton and a second `init` call throws.
168+
- If `@opentelemetry/api` is not installed, `init` throws.
169+
170+
## Runnable example
171+
172+
See ../examples/otel-metrics.js.

‎examples/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ This folder contains example scripts showing how to use Node Redis in different
1818
| `hyperloglog.js` | Showing use of Hyperloglog commands [PFADD, PFCOUNT and PFMERGE](https://redis.io/commands/?group=hyperloglog). |
1919
| `lua-multi-incr.js` | Define a custom lua script that allows you to perform INCRBY on multiple keys. |
2020
| `managing-json.js` | Store, retrieve and manipulate JSON data atomically with [RedisJSON](https://redisjson.io/). |
21+
| `otel-metrics.js` | Enable OpenTelemetry metrics for node-redis, generate command and resiliency signals, and export them via OpenTelemetry SDK metrics. |
2122
| `pubsub-publisher.js` | Adds multiple messages on 2 different channels messages to Redis. |
2223
| `pubsub-subscriber.js` | Reads messages from channels using `PSUBSCRIBE` command. |
2324
| `search-hashes.js` | Uses [RediSearch](https://redisearch.io) to index and search data in hashes. |

‎examples/otel-metrics.js‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
// OpenTelemetry metrics example for node-redis.
2+
// Demonstrates enablement, command activity, and metric export verification.
3+
4+
import { createClient, OpenTelemetry } from 'redis';
5+
import { metrics } from '@opentelemetry/api';
6+
import {
7+
ConsoleMetricExporter,
8+
MeterProvider,
9+
PeriodicExportingMetricReader
10+
} from '@opentelemetry/sdk-metrics';
11+
12+
// Export metrics to console for easy local verification.
13+
const reader = new PeriodicExportingMetricReader({
14+
exporter: new ConsoleMetricExporter(),
15+
});
16+
17+
const meterProvider = new MeterProvider({
18+
readers: [reader]
19+
});
20+
21+
metrics.setGlobalMeterProvider(meterProvider);
22+
23+
// Enable OpenTelemetry before creating clients
24+
OpenTelemetry.init({
25+
metrics: {
26+
enabled: true,
27+
enabledMetricGroups: ['command', 'connection-basic', 'resiliency'],
28+
}
29+
});
30+
31+
const client = createClient();
32+
33+
client.on('error', (err) => {
34+
console.error('Redis client error:', err);
35+
});
36+
37+
try {
38+
await client.connect();
39+
40+
// Normal command traffic.
41+
await client.ping();
42+
await client.set('otel:example:key', '1');
43+
await client.get('otel:example:key');
44+
45+
// Generate a handled error to demonstrate resiliency metrics.
46+
await client.hSet('otel:example:hash', 'field', 'value');
47+
try {
48+
await client.incr('otel:example:hash');
49+
} catch (err) {
50+
console.log('Expected command error:', err);
51+
}
52+
53+
// Force export so output is visible immediately.
54+
await meterProvider.forceFlush();
55+
} finally {
56+
client.destroy();
57+
await meterProvider.shutdown();
58+
}

0 commit comments

Comments
 (0)