Skip to content

Commit 2710fe9

Browse files
hhertouthhertout
committed
feat(otelcol.connector): adding connector count
1 parent af1c3df commit 2710fe9

File tree

11 files changed

+1152
-26
lines changed

11 files changed

+1152
-26
lines changed

‎CHANGELOG.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,8 @@ Main (unreleased)
1212

1313
### Features
1414

15+
- Add `otelcol.connector.count` component to count the number of spans, metrics, and logs passing through it. (@hhertout)
16+
1517
- A new `mimir.alerts.kubernetes` component which discovers `AlertmanagerConfig` Kubernetes resources and loads them into a Mimir instance. (@ptodev)
1618

1719
- Mark `stage.windowsevent` block in the `loki.process` component as GA. (@kgeckhart)

‎docs/sources/reference/components/otelcol/otelcol.connector.count.md‎

Lines changed: 370 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,370 @@
1+
---
2+
canonical: https://grafana.com/docs/alloy/latest/reference/components/otelcol/otelcol.connector.count/
3+
aliases:
4+
- ../otelcol.connector.count/ # /docs/alloy/latest/reference/components/otelcol.connector.count/
5+
description: Learn about otelcol.connector.count
6+
labels:
7+
stage: experimental
8+
products:
9+
- oss
10+
title: otelcol.connector.count
11+
---
12+
13+
# `otelcol.connector.count`
14+
15+
`otelcol.connector.count` accepts spans, span events, metrics, data points, and log records from other `otelcol` components and generates metrics that count the received telemetry data.
16+
17+
{{< admonition type="note" >}}
18+
`otelcol.connector.count` is a wrapper over the upstream OpenTelemetry Collector Contrib `count` connector.
19+
Bug reports or feature requests are redirected to the upstream repository if necessary.
20+
{{< /admonition >}}
21+
22+
You can specify multiple `otelcol.connector.count` components by giving them different labels.
23+
24+
## Usage
25+
26+
```alloy
27+
otelcol.connector.count "<LABEL>" {
28+
output {
29+
metrics = [...]
30+
}
31+
}
32+
```
33+
34+
## Arguments
35+
36+
`otelcol.connector.count` doesn't support any arguments and can be configured using blocks only.
37+
38+
## Blocks
39+
40+
You can use the following blocks with `otelcol.connector.count`:
41+
42+
| Block | Description | Required |
43+
| -------------------------------- | ---------------------------------------------------------------------------------------- | -------- |
44+
| [`spans`][spans] | Configures custom metrics for counting spans. | no |
45+
| [`spans` > `attributes`][attributes] | Groups span counts by attribute values. | no |
46+
| [`spanevents`][spanevents] | Configures custom metrics for counting span events. | no |
47+
| [`spanevents` > `attributes`][attributes] | Groups span event counts by attribute values. | no |
48+
| [`metrics`][metrics] | Configures custom metrics for counting metrics. | no |
49+
| [`metrics` > `attributes`][attributes] | Groups metric counts by attribute values. | no |
50+
| [`datapoints`][datapoints] | Configures custom metrics for counting data points. | no |
51+
| [`datapoints` > `attributes`][attributes] | Groups data point counts by attribute values. | no |
52+
| [`logs`][logs] | Configures custom metrics for counting log records. | no |
53+
| [`logs` > `attributes`][attributes] | Groups log counts by attribute values. | no |
54+
| [`output`][output] | Configures where to send received telemetry data. | yes |
55+
| [`debug_metrics`][debug_metrics] | Configures the metrics that this component generates to monitor its state. | no |
56+
57+
The `>` symbol indicates deeper levels of nesting.
58+
For example, `spans` > `attributes` refers to an `attributes` block defined inside a `spans` block.
59+
60+
[spans]: #spans
61+
[spanevents]: #spanevents
62+
[metrics]: #metrics
63+
[datapoints]: #datapoints
64+
[logs]: #logs
65+
[attributes]: #attributes
66+
[output]: #output
67+
[debug_metrics]: #debug_metrics
68+
69+
### `spans`
70+
71+
The `spans` block configures a custom metric for counting spans.
72+
73+
You can specify multiple `spans` blocks to define different metrics.
74+
75+
If no `spans` blocks are defined, the connector emits a default metric named `trace.span.count`.
76+
77+
Name | Type | Description | Default | Required
78+
---- | ---- | ----------- | ------- | --------
79+
`name` | `string` | Metric name. | Uses default name based on telemetry type. | no
80+
`description` | `string` | Metric description. | `""` | no
81+
`conditions` | `list(string)` | OTTL expressions for filtering spans. | `[]` | no
82+
83+
The `conditions` argument accepts a list of [OTTL][] expressions that filter which spans to count.
84+
If any condition matches, the span is counted.
85+
If no conditions are specified, all spans are counted.
86+
87+
[OTTL]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl
88+
89+
### `spanevents`
90+
91+
The `spanevents` block configures a custom metric for counting span events.
92+
93+
You can specify multiple `spanevents` blocks to define different metrics.
94+
95+
If no `spanevents` blocks are defined, the connector emits a default metric named `trace.span.event.count`.
96+
97+
This block shares the same configuration structure as [`spans`][spans].
98+
Refer to the [`spans`][spans] block documentation for the complete list of supported arguments and blocks.
99+
100+
### `metrics`
101+
102+
The `metrics` block configures a custom metric for counting metrics.
103+
104+
You can specify multiple `metrics` blocks to define different metrics.
105+
106+
If no `metrics` blocks are defined, the connector emits a default metric named `metric.count`.
107+
108+
This block shares the same configuration structure as [`spans`][spans].
109+
Refer to the [`spans`][spans] block documentation for the complete list of supported arguments and blocks.
110+
111+
### `datapoints`
112+
113+
The `datapoints` block configures a custom metric for counting data points.
114+
115+
You can specify multiple `datapoints` blocks to define different metrics.
116+
117+
If no `datapoints` blocks are defined, the connector emits a default metric named `metric.datapoint.count`.
118+
119+
This block shares the same configuration structure as [`spans`][spans].
120+
Refer to the [`spans`][spans] block documentation for the complete list of supported arguments and blocks.
121+
122+
### `logs`
123+
124+
The `logs` block configures a custom metric for counting log records.
125+
126+
You can specify multiple `logs` blocks to define different metrics.
127+
128+
If no `logs` blocks are defined, the connector emits a default metric named `log.record.count`.
129+
130+
This block shares the same configuration structure as [`spans`][spans].
131+
Refer to the [`spans`][spans] block documentation for the complete list of supported arguments and blocks.
132+
133+
### `attributes`
134+
135+
The `attributes` block specifies an attribute to use for grouping counted telemetry data.
136+
137+
Each unique combination of attribute values generates a separate data point on the metric.
138+
139+
You can specify multiple `attributes` blocks within `spans`, `spanevents`, `metrics`, `datapoints`, or `logs` blocks to group by multiple attributes.
140+
141+
The following arguments are supported:
142+
143+
Name | Type | Description | Default | Required
144+
---- | ---- | ----------- | ------- | --------
145+
`key` | `string` | Attribute key name. | | yes
146+
`default_value` | `any` | Default value if the attribute doesn't exist. | | no
147+
148+
Attribute precedence: span/log/metric attributes > scope attributes > resource attributes.
149+
150+
### `output`
151+
152+
{{< docs/shared lookup="reference/components/output-block.md" source="alloy" version="<ALLOY_VERSION>" >}}
153+
154+
### `debug_metrics`
155+
156+
{{< docs/shared lookup="reference/components/otelcol-debug-metrics-block.md" source="alloy" version="<ALLOY_VERSION>" >}}
157+
158+
## Exported fields
159+
160+
The following fields are exported and can be referenced by other components:
161+
162+
| Name | Type | Description |
163+
| ------- | ------------------ | ---------------------------------------------------------------- |
164+
| `input` | `otelcol.Consumer` | A value that other components can use to send telemetry data to. |
165+
166+
`input` accepts `otelcol.Consumer` data for traces, metrics, and logs.
167+
168+
All telemetry received through `input` is counted according to the configured blocks and emitted as metrics to the components specified in `output`.
169+
170+
## Component health
171+
172+
`otelcol.connector.count` is only reported as unhealthy if given an invalid configuration.
173+
174+
## Debug information
175+
176+
`otelcol.connector.count` doesn't expose any component-specific debug information.
177+
178+
## Examples
179+
180+
### Default configuration
181+
182+
Use the count connector with minimal configuration to count all telemetry data using default metric names.
183+
184+
```alloy
185+
otelcol.connector.count "default" {
186+
output {
187+
metrics = [otelcol.exporter.otlp.default.input]
188+
}
189+
}
190+
```
191+
192+
This configuration generates the following default metrics:
193+
194+
- `trace.span.count` - Count of all spans
195+
- `trace.span.event.count` - Count of all span events
196+
- `metric.count` - Count of all metrics
197+
- `metric.datapoint.count` - Count of all data points
198+
- `log.record.count` - Count of all log records
199+
200+
### Custom metrics with conditions and attributes
201+
202+
Create custom count metrics with filtering conditions and group counts by specific attributes.
203+
204+
```alloy
205+
otelcol.connector.count "default" {
206+
// Count only HTTP GET spans
207+
spans {
208+
name = "http_get_requests"
209+
description = "Number of HTTP GET requests"
210+
conditions = [
211+
"attributes[\"http.method\"] == \"GET\"",
212+
]
213+
}
214+
215+
// Count spans grouped by service and environment
216+
spans {
217+
name = "spans_by_service"
218+
description = "Spans per service and environment"
219+
attributes {
220+
key = "service.name"
221+
default_value = "unknown"
222+
}
223+
attributes {
224+
key = "env"
225+
default_value = "production"
226+
}
227+
}
228+
229+
// Count error and fatal logs only
230+
logs {
231+
name = "error_logs"
232+
description = "Error and fatal log records"
233+
conditions = [
234+
"severity_number >= 17",
235+
]
236+
}
237+
238+
// Count logs by environment
239+
logs {
240+
name = "logs_by_env"
241+
description = "Log records per environment"
242+
attributes {
243+
key = "env"
244+
}
245+
}
246+
247+
output {
248+
metrics = [otelcol.exporter.otlp.default.input]
249+
}
250+
}
251+
```
252+
253+
### Complete pipeline with Prometheus export
254+
255+
Count spans, logs, and metrics, then export to Prometheus using the delta to cumulative processor.
256+
257+
```alloy
258+
otelcol.receiver.otlp "default" {
259+
grpc {
260+
endpoint = "127.0.0.1:4317"
261+
}
262+
263+
http {
264+
endpoint = "127.0.0.1:4318"
265+
}
266+
267+
output {
268+
traces = [otelcol.connector.count.default.input]
269+
metrics = [otelcol.connector.count.default.input]
270+
logs = [otelcol.connector.count.default.input]
271+
}
272+
}
273+
274+
otelcol.connector.count "default" {
275+
spans {
276+
name = "traces_total"
277+
description = "Total number of spans received"
278+
}
279+
280+
spans {
281+
name = "http_get_spans"
282+
description = "HTTP GET requests"
283+
conditions = [
284+
"attributes[\"http.method\"] == \"GET\"",
285+
]
286+
}
287+
288+
spans {
289+
name = "spans_by_service"
290+
description = "Spans grouped by service"
291+
attributes {
292+
key = "service.name"
293+
default_value = "unknown"
294+
}
295+
}
296+
297+
logs {
298+
name = "logs_total"
299+
description = "Total number of logs received"
300+
}
301+
302+
logs {
303+
name = "error_logs"
304+
description = "Error level logs"
305+
conditions = [
306+
"severity_number >= 17",
307+
]
308+
}
309+
310+
metrics {
311+
name = "metrics_total"
312+
description = "Total number of metrics received"
313+
}
314+
315+
output {
316+
metrics = [otelcol.processor.deltatocumulative.default.input]
317+
}
318+
}
319+
320+
// Convert delta metrics to cumulative for Prometheus compatibility
321+
otelcol.processor.deltatocumulative "default" {
322+
output {
323+
metrics = [otelcol.exporter.prometheus.default.input]
324+
}
325+
}
326+
327+
otelcol.exporter.prometheus "default" {
328+
forward_to = [prometheus.remote_write.default.receiver]
329+
add_metric_suffixes = false
330+
resource_to_telemetry_conversion = true
331+
}
332+
333+
prometheus.remote_write "default" {
334+
endpoint {
335+
url = "http://localhost:9090/api/v1/write"
336+
}
337+
}
338+
```
339+
340+
## Technical details
341+
342+
`otelcol.connector.count` uses the count connector from OpenTelemetry Collector Contrib.
343+
344+
All generated metrics use the Sum data type with Delta aggregation temporality.
345+
346+
{{< admonition type="note" >}}
347+
Prometheus doesn't natively support delta metrics.
348+
Use [`otelcol.processor.deltatocumulative`][otelcol.processor.deltatocumulative] to convert delta metrics to cumulative before sending to Prometheus.
349+
350+
[otelcol.processor.deltatocumulative]: ../otelcol.processor.deltatocumulative/
351+
{{< /admonition >}}
352+
353+
<!-- START GENERATED COMPATIBLE COMPONENTS -->
354+
355+
## Compatible components
356+
357+
`otelcol.connector.count` can accept arguments from the following components:
358+
359+
- Components that export [OpenTelemetry `otelcol.Consumer`](../../../compatibility/#opentelemetry-otelcolconsumer-exporters)
360+
361+
`otelcol.connector.count` has exports that can be consumed by the following components:
362+
363+
- Components that consume [OpenTelemetry `otelcol.Consumer`](../../../compatibility/#opentelemetry-otelcolconsumer-consumers)
364+
365+
{{< admonition type="note" >}}
366+
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly.
367+
Refer to the linked documentation for more details.
368+
{{< /admonition >}}
369+
370+
<!-- END GENERATED COMPATIBLE COMPONENTS -->

‎go.mod‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,7 @@ require (
116116
github.com/oklog/run v1.2.0
117117
github.com/olekukonko/tablewriter v0.0.5
118118
github.com/oliver006/redis_exporter v1.74.0
119+
github.com/open-telemetry/opentelemetry-collector-contrib/connector/countconnector v0.139.0
119120
github.com/open-telemetry/opentelemetry-collector-contrib/connector/servicegraphconnector v0.139.0
120121
github.com/open-telemetry/opentelemetry-collector-contrib/connector/spanmetricsconnector v0.139.0
121122
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/awss3exporter v0.139.0

‎go.sum‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1755,6 +1755,8 @@ github.com/onsi/ginkgo/v2 v2.23.4 h1:ktYTpKJAVZnDT4VjxSbiBenUjmlL/5QkBEocaWXiQus
17551755
github.com/onsi/ginkgo/v2 v2.23.4/go.mod h1:Bt66ApGPBFzHyR+JO10Zbt0Gsp4uWxu5mIOTusL46e8=
17561756
github.com/onsi/gomega v1.37.0 h1:CdEG8g0S133B4OswTDC/5XPSzE1OeP29QOioj2PID2Y=
17571757
github.com/onsi/gomega v1.37.0/go.mod h1:8D9+Txp43QWKhM24yyOBEdpkzN8FvJyAwecBgsU4KU0=
1758+
github.com/open-telemetry/opentelemetry-collector-contrib/connector/countconnector v0.139.0 h1:L/mt81EynyqP8+Eo1RScotNMKodU8tzu5kdDVZsxzos=
1759+
github.com/open-telemetry/opentelemetry-collector-contrib/connector/countconnector v0.139.0/go.mod h1:aamgZ/EgklfjbFzEpZHJFw6558jhx7iVrH/BF4aIkDM=
17581760
github.com/open-telemetry/opentelemetry-collector-contrib/connector/servicegraphconnector v0.139.0 h1:wsylHna5nuu6Jl+ow9uv3WJ0qKGJxg2aRORFUDa8SAs=
17591761
github.com/open-telemetry/opentelemetry-collector-contrib/connector/servicegraphconnector v0.139.0/go.mod h1:LIR7KA0MIoHFQN/mO3zLEeK7piZL3cR6Ig10GfpS5UU=
17601762
github.com/open-telemetry/opentelemetry-collector-contrib/connector/spanmetricsconnector v0.139.0 h1:USQU4VEL4Vi1rDm1am6LFjIvRGSOWhb+huw1OLIo3Eo=

0 commit comments

Comments
 (0)