Skip to content

feat: add metrics backwards-compatibility CI framework#8204

Draft
Amaan729 wants to merge 2 commits intojaegertracing:mainfrom
Amaan729:feat/metrics-backwards-compat
Draft

feat: add metrics backwards-compatibility CI framework#8204
Amaan729 wants to merge 2 commits intojaegertracing:mainfrom
Amaan729:feat/metrics-backwards-compat

Conversation

@Amaan729
Copy link

Closes #6278

Summary

Implements the first incremental slice of the metrics backwards-compatibility framework requested in #6278.

What this PR adds

.github/actions/verify-metrics-snapshot/ — new composite action

A reusable GitHub Actions composite action that:

  1. Waits for Jaeger to start (retry loop), then curls the /metrics endpoint.
  2. Uploads the raw Prometheus text as a GitHub artifact (keyed per storage backend, retained 30 days).
  3. Diffs the live metric names against a committed baseline snapshot and fails CI if any metric was removed or renamed (additions are allowed and logged as a notice).

The action is already referenced in ci-e2e-memory.yaml via:

- uses: ./.github/actions/verify-metrics-snapshot
  with:
    snapshot: metrics_snapshot_memory
    artifact_key: metrics_snapshot_memory

testdata/metrics/ — baseline snapshot directory

Empty directory with a README explaining the workflow. Baseline .txt files are committed here after the first successful CI run.

What's intentionally out of scope for this PR

  • Wiring the action into the remaining e2e workflows (cassandra, ES, OS, …) — follow-up PR.
  • Downloading the previous release's artifact and diffing against it — follow-up PR once baselines are established.
  • A Go script as an alternative scraper — the curl-based approach is simpler and has no build-time dependency.

How to establish the first baseline

After this PR is merged and a workflow run completes:

# Download the artifact from the Actions UI, then:
cp metrics_snapshot_memory.txt testdata/metrics/metrics_snapshot_memory.txt
git add testdata/metrics/metrics_snapshot_memory.txt
git commit -m "chore: add initial metrics snapshot baseline for memory backend"

Testing

  • Action YAML syntax validated locally with actionlint.
  • Logic tested by running the shell snippet manually against a local Jaeger instance.

Checklist

  • Added composite action
  • Added baseline directory + README
  • Incremental scope (does not try to do everything in one PR)
  • Baseline .txt files (will be committed in a follow-up after first CI run)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement framework to validate backwards compatibility of metrics

1 participant