Skip to content

tsdb: add an hourly rollup tier for the cluster table (tsdb_metrics_cluster_hour) #6072

Description

@renecannao

PR #6034 replicates peers' raw TSDB samples into tsdb_metrics_cluster on the leader, with no rollup tier — v1 replicates raw only.

Measured cost (2026-08-13 lab run, 417-series real fixture): the cluster tier is ~104 B/row and a leader carries roughly 7.8 GB total (~2.3x a follower) at the 24h/14d profile, of which ~4.5 GB is cluster raw. Long-horizon node= queries are raw scans, and cluster raw retention must stay short (now 1 day) purely to bound size — which means the cluster view has no long-horizon data at all.

Design sketch (from the spec's follow-ups):

  1. Leader downsamples its own tsdb_metrics_cluster into tsdb_metrics_cluster_hour keyed by (node, bucket, metric_name, labels), with its own retention variable. Self-sufficient, no protocol change.
  2. Later: pull peers' tsdb_metrics_hour directly — the only way to obtain cluster history older than the backfill horizon at the moment a node first becomes leader.

Test bed exists: test/tsdb-lab/expand.py --nodes N produces a 14-day cluster table on demand.

Needs a design round (spec -> plan) before implementation.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions