Skip to content

Commit b9f7993

Browse files
hharshhsainihharshhsaini
committed
feat(docs): add v2 metrics reference for backwards compatibility tracking
Signed-off-by: hharshhsaini <sainiharsh3311@gmail.com>
1 parent fa2e147 commit b9f7993

File tree

6 files changed

+313
-0
lines changed

6 files changed

+313
-0
lines changed

‎.cspell/project-words.txt‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ hugo
4747
ingester
4848
ingester's
4949
ingesters
50+
inuse
5051
istio
5152
istio's
5253
jaegertracing
@@ -64,14 +65,19 @@ logtostderr
6465
logz
6566
lookback
6667
mailgroup
68+
mallocs
6769
markdownify
70+
mcache
71+
memstats
72+
memtable
6873
mentee
6974
mentees
7075
mentorships
7176
metricsquery
7277
metricsstore
7378
metricstore
7479
microsim
80+
mspan
7581
myapp
7682
myappnamespace
7783
mynamespace
@@ -132,6 +138,7 @@ unmarshaling
132138
upsample
133139
usercert
134140
userkey
141+
valueloggc
135142
valuen
136143
vlog
137144
warnf

‎.htmltest.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ IgnoreURLs:
1111
- ^https://www.jaegertracing.io/js/lunr
1212
# Ignore Docsy-generated GitHub links for now
1313
- ^https?://github\.com/.*?/.*?/(new|edit)/ # view-page, edit-source etc
14+
- ^https://github\.com/jaegertracing/documentation/tree/main/content/docs/v2/_dev/operations/metrics-reference\.md
1415
# Valid URL, whether canonical (before reorg) or not (and handled via a redirect)
1516
- ^/docs/1.[6-9]\d/sampling/
1617
# Valid URLs, but servers yield 403 or similar errors

‎Makefile‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,9 @@ fetch-blog-feed:
6565
fetch-roadmap:
6666
python3 scripts/generate_roadmap.py
6767

68+
fetch-metrics:
69+
python3 scripts/fetch_metrics_reference.py
70+
6871
# only x.y.0 semver values are valid for kicking off a new release.
6972
SEMVER_REGEX := ^([0-9]+\.){2}0$$
7073
VALID_VERSION := $(shell echo "$(VERSION)" | grep -E "$(SEMVER_REGEX)")

‎content/docs/v2/_dev/operations/_index.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ children:
1010
url: performance-tuning
1111
- title: Tools
1212
url: tools
13+
- title: Metrics Reference (v2)
14+
url: metrics-reference
1315
---
1416

1517
Jaeger provides the foundational components to operate your installation. Please note that backups and lifecycle of the datastores must be handled outside of Jaeger. We provide pointers to components within the [storage backends](../architecture/) guides.

‎content/docs/v2/_dev/operations/metrics-reference.md‎

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
---
2+
title: Metrics Reference (v2)
3+
type: docs
4+
weight: 80
5+
---
6+
7+
This page lists all metrics currently exposed by Jaeger v2. These metrics are automatically tracked by the Jaeger CI pipeline to ensure **backwards compatibility across Jaeger v2 releases** — i.e. that metric names and labels do not change unexpectedly between v2 versions.
8+
9+
> **Note:** This page is auto-generated. Run `make fetch-metrics` to update it.
10+
11+
## All-in-One Component Metrics
12+
13+
### ALL-IN-ONE METRICS
14+
### Combined Metrics
15+
| Metric | Labels |
16+
|--------|--------|
17+
| jaeger_query_latency | operation, result |
18+
| jaeger_query_responses | operation |
19+
| jaeger_query_requests_total | operation, result |
20+
| exporter_send_failed_spans | exporter, service_instance_id, service_name, service_version |
21+
| exporter_sent_spans | exporter, service_instance_id, service_name, service_version |
22+
| process_cpu_seconds | service_instance_id, service_name, service_version |
23+
| process_memory_rss | service_instance_id, service_name, service_version |
24+
| process_runtime_heap_alloc_bytes | service_instance_id, service_name, service_version |
25+
| process_runtime_total_alloc_bytes | service_instance_id, service_name, service_version |
26+
| process_runtime_total_sys_memory_bytes | service_instance_id, service_name, service_version |
27+
| process_uptime | service_instance_id, service_name, service_version |
28+
| processor_batch_batch_send_size | processor, service_instance_id, service_name, service_version |
29+
| processor_batch_batch_send_size_bytes | processor, service_instance_id, service_name, service_version |
30+
| processor_batch_metadata_cardinality | processor, service_instance_id, service_name, service_version |
31+
| processor_batch_timeout_trigger_send | processor, service_instance_id, service_name, service_version |
32+
| receiver_accepted_spans | receiver, service_instance_id, service_name, service_version, transport |
33+
| receiver_refused_spans | receiver, service_instance_id, service_name, service_version, transport |
34+
| rpc_server_duration | rpc_grpc_status_code, rpc_method, rpc_service, rpc_system, service_instance_id, service_name, service_version |
35+
| rpc_server_request_size | rpc_method, rpc_service, rpc_system, service_instance_id, service_name, service_version |
36+
| rpc_server_requests_per_rpc | rpc_grpc_status_code, rpc_method, rpc_service, rpc_system, service_instance_id, service_name, service_version |
37+
| rpc_server_response_size | rpc_method, rpc_service, rpc_system, service_instance_id, service_name, service_version |
38+
| rpc_server_responses_per_rpc | rpc_grpc_status_code, rpc_method, rpc_service, rpc_system, service_instance_id, service_name, service_version |
39+
| target_info | service_instance_id, service_name, service_version |
40+
41+
| Metric | Labels |
42+
|--------|--------|
43+
| receiver_refused_spans | receiver, service_instance_id, service_name, service_version, transport |
44+
| target_info | service_instance_id, service_name, service_version |
45+
46+
## Badger Storage Metrics
47+
48+
### BADGER METRICS
49+
### Combined Metrics
50+
| Metric | Labels |
51+
|--------|--------|
52+
| jaeger_badger_compaction_current_num_lsm | N/A |
53+
| jaeger_badger_get_num_memtable | N/A |
54+
| jaeger_badger_get_num_user | N/A |
55+
| jaeger_badger_get_with_result_num_user | N/A |
56+
| jaeger_badger_iterator_num_user | N/A |
57+
| jaeger_badger_put_num_user | N/A |
58+
| jaeger_badger_read_bytes_lsm | N/A |
59+
| jaeger_badger_read_bytes_vlog | N/A |
60+
| jaeger_badger_read_num_vlog | N/A |
61+
| jaeger_badger_size_bytes_lsm | N/A |
62+
| jaeger_badger_size_bytes_vlog | N/A |
63+
| jaeger_badger_write_bytes_l0 | N/A |
64+
| jaeger_badger_write_bytes_user | N/A |
65+
| jaeger_badger_write_bytes_vlog | N/A |
66+
| jaeger_badger_write_num_vlog | N/A |
67+
| jaeger_badger_write_pending_num_memtable | N/A |
68+
69+
### Equivalent Metrics
70+
| Metric | Labels |
71+
|--------|--------|
72+
| receiver_refused_spans | receiver, service_instance_id, service_name, service_version, transport |
73+
| target_info | service_instance_id, service_name, service_version |
74+
75+
## Cassandra Storage Metrics
76+
77+
### CASSANDRA METRICS
78+
### Combined Metrics
79+
| Metric | Labels |
80+
|--------|--------|
81+
| jaeger_cassandra_attempts_total | table |
82+
| jaeger_cassandra_errors_total | table |
83+
| jaeger_cassandra_inserts_total | table |
84+
| jaeger_cassandra_latency_err | table |
85+
| jaeger_cassandra_latency_ok | table |
86+
| jaeger_cassandra_read_attempts_total | table |
87+
| jaeger_cassandra_read_errors_total | table |
88+
| jaeger_cassandra_read_inserts_total | table |
89+
| jaeger_cassandra_read_latency_err | table |
90+
| jaeger_cassandra_read_latency_ok | table |
91+
| jaeger_cassandra_tag_index_skipped_total | N/A |
92+
93+
| Metric | Labels |
94+
|--------|--------|
95+
| receiver_refused_spans | receiver, service_instance_id, service_name, service_version, transport |
96+
| target_info | service_instance_id, service_name, service_version |
97+
98+
## Elasticsearch Storage Metrics
99+
100+
### ELASTICSEARCH METRICS
101+
### Combined Metrics
102+
| Metric | Labels |
103+
|--------|--------|
104+
| jaeger_bulk_index_attempts_total | N/A |
105+
| jaeger_bulk_index_errors_total | N/A |
106+
| jaeger_bulk_index_inserts_total | N/A |
107+
| jaeger_bulk_index_latency_err | N/A |
108+
| jaeger_bulk_index_latency_ok | N/A |
109+
| jaeger_index_create_attempts_total | N/A |
110+
| jaeger_index_create_errors_total | N/A |
111+
| jaeger_index_create_inserts_total | N/A |
112+
| jaeger_index_create_latency_err | N/A |
113+
| jaeger_index_create_latency_ok | N/A |
114+
115+
| Metric | Labels |
116+
|--------|--------|
117+
| receiver_refused_spans | receiver, service_instance_id, service_name, service_version, transport |
118+
| target_info | service_instance_id, service_name, service_version |
119+
120+
## OpenSearch Storage Metrics
121+
122+
### OPENSEARCH METRICS
123+
### Combined Metrics
124+
| Metric | Labels |
125+
|--------|--------|
126+
| jaeger_bulk_index_attempts_total | N/A |
127+
| jaeger_bulk_index_errors_total | N/A |
128+
| jaeger_bulk_index_inserts_total | N/A |
129+
| jaeger_bulk_index_latency_err | N/A |
130+
| jaeger_bulk_index_latency_ok | N/A |
131+
| jaeger_index_create_attempts_total | N/A |
132+
| jaeger_index_create_errors_total | N/A |
133+
| jaeger_index_create_inserts_total | N/A |
134+
| jaeger_index_create_latency_err | N/A |
135+
| jaeger_index_create_latency_ok | N/A |
136+
137+
| Metric | Labels |
138+
|--------|--------|
139+
| receiver_refused_spans | receiver, service_instance_id, service_name, service_version, transport |
140+
| target_info | service_instance_id, service_name, service_version |
141+

‎scripts/fetch_metrics_reference.py‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
#!/usr/bin/env python3
2+
3+
# Copyright (c) 2024 The Jaeger Authors.
4+
# SPDX-License-Identifier: Apache-2.0
5+
6+
# This script fetches the auto-generated metrics reports from the jaegertracing/jaeger
7+
# repository and compiles a reference page of current Jaeger v2 metrics.
8+
# The purpose is to track BACKWARDS COMPATIBILITY of metrics across Jaeger v2 releases.
9+
10+
import logging
11+
import os
12+
import re
13+
import sys
14+
import urllib.error
15+
import urllib.request
16+
from pathlib import Path
17+
18+
# Set up logging
19+
logging.basicConfig(level=logging.INFO)
20+
logger = logging.getLogger(__name__)
21+
22+
# Base URL for the raw markdown files generated by the jaeger E2E workflow
23+
BASE_RAW_URL = "https://raw.githubusercontent.com/jaegertracing/jaeger/main/cmd/jaeger/docs/migration/"
24+
25+
TARGET_FILES = [
26+
{"filename": "all-in-one-metrics.md", "title": "All-in-One Component"},
27+
{"filename": "badger-metrics.md", "title": "Badger Storage"},
28+
{"filename": "cassandra-metrics.md", "title": "Cassandra Storage"},
29+
{"filename": "elasticsearch-metrics.md", "title": "Elasticsearch Storage"},
30+
{"filename": "opensearch-metrics.md", "title": "OpenSearch Storage"},
31+
]
32+
33+
34+
def fetch_markdown(filename):
35+
url = BASE_RAW_URL + filename
36+
req = urllib.request.Request(url)
37+
try:
38+
with urllib.request.urlopen(req, timeout=10) as response:
39+
return response.read().decode("utf-8")
40+
except urllib.error.HTTPError as e:
41+
logger.error(f"HTTP Error fetching {filename}: {e.code} {e.reason}")
42+
return None
43+
except Exception as e:
44+
logger.error(f"Error fetching {filename}: {e}")
45+
return None
46+
47+
48+
def extract_v2_metrics_table(raw_md):
49+
"""Parse the raw markdown and return a new table with only v2 Metric + v2 Labels columns.
50+
51+
The source files have 4 columns: V1 Metric | V1 Labels | V2 Metric | V2 Labels.
52+
We extract only the columns that matter for v2 backwards-compatibility tracking,
53+
dropping rows where V2 Metric is N/A (metric was removed in v2).
54+
"""
55+
output_lines = []
56+
in_table = False
57+
58+
for line in raw_md.split("\n"):
59+
stripped = line.strip()
60+
61+
# Detect the start of a 4-column metrics table (V1 Metric header)
62+
if "V1 Metric" in stripped and "V2 Metric" in stripped:
63+
in_table = True
64+
# Emit the new v2-only header
65+
output_lines.append("| Metric | Labels |")
66+
output_lines.append("|--------|--------|")
67+
continue
68+
69+
# Skip the separator row (|---|---:|:---|...)
70+
if in_table and re.fullmatch(r'[\|\-\s:]+', stripped):
71+
continue
72+
73+
if in_table and stripped.startswith("|"):
74+
# Strip leading/trailing pipes first to avoid empty first column from `|| ...` rows
75+
cols = [c.strip() for c in stripped.strip('|').split("|")]
76+
# cols[0]=V1 Metric, cols[1]=V1 Labels, cols[2]=V2 Metric, cols[3]=V2 Labels
77+
if len(cols) >= 4:
78+
v2_metric = cols[2]
79+
v2_labels = cols[3]
80+
# Skip metrics that don't exist in v2
81+
if v2_metric and v2_metric.lower() != "n/a":
82+
output_lines.append(f"| {v2_metric} | {v2_labels} |")
83+
elif in_table and stripped == "":
84+
in_table = False
85+
output_lines.append("")
86+
elif not in_table:
87+
# Carry over subsection headings, but downgrade H1 to H3 to avoid multiple page H1s.
88+
# Preserve H2-H6 headings as-is to keep section boundaries clear.
89+
heading_match = re.match(r'^(#{1,6})\s+(.*)', stripped)
90+
if heading_match:
91+
hashes, text = heading_match.groups()
92+
level = len(hashes)
93+
if level == 1:
94+
# Downgrade H1 to H3
95+
output_lines.append("### " + text.strip())
96+
else:
97+
# Preserve H2-H6 verbatim
98+
output_lines.append(stripped)
99+
100+
return "\n".join(output_lines)
101+
102+
103+
def generate_metrics_reference():
104+
content = "---\n"
105+
content += "title: Metrics Reference (v2)\n"
106+
content += "type: docs\n"
107+
content += "weight: 80\n"
108+
content += "---\n\n"
109+
content += (
110+
"This page lists all metrics currently exposed by Jaeger v2. "
111+
"These metrics are automatically tracked by the Jaeger CI pipeline to ensure "
112+
"**backwards compatibility across Jaeger v2 releases** — i.e. that metric names and labels "
113+
"do not change unexpectedly between v2 versions.\n\n"
114+
)
115+
content += "> **Note:** This page is auto-generated. Run `make fetch-metrics` to update it.\n\n"
116+
117+
missing_files = False
118+
119+
for item in TARGET_FILES:
120+
logger.info(f"Fetching {item['filename']}...")
121+
raw_md = fetch_markdown(item["filename"])
122+
if raw_md is not None:
123+
v2_table = extract_v2_metrics_table(raw_md)
124+
content += f"## {item['title']} Metrics\n\n"
125+
content += v2_table.strip() + "\n\n"
126+
else:
127+
logger.warning(f"Skipping {item['filename']} due to fetch failure.")
128+
missing_files = True
129+
130+
return content, missing_files
131+
132+
133+
def save_metrics_reference(content):
134+
# Resolve output path relative to the repository root (parent of the scripts/ dir)
135+
# so the script works correctly regardless of the current working directory.
136+
repo_root = Path(__file__).resolve().parent.parent
137+
target_path = repo_root / "content/docs/v2/_dev/operations/metrics-reference.md"
138+
os.makedirs(target_path.parent, exist_ok=True)
139+
with open(target_path, "w", encoding="utf-8") as f:
140+
f.write(content)
141+
logger.info(f"Successfully saved metrics reference to {target_path}")
142+
143+
144+
def main():
145+
try:
146+
content, missing_files = generate_metrics_reference()
147+
save_metrics_reference(content)
148+
149+
if missing_files:
150+
logger.error("Failed to fetch one or more metrics files. The generated page may be incomplete.")
151+
sys.exit(1)
152+
153+
except Exception as e:
154+
logger.error(f"An error occurred: {e}")
155+
sys.exit(1)
156+
157+
158+
if __name__ == "__main__":
159+
main()

0 commit comments

Comments
 (0)