Skip to content

fix: Initialize metrics on startup to fix restart gap; ProbeTx can be disabled#6

Open
KA-ROM wants to merge 8 commits intomainfrom
metric_init_with_0
Open

fix: Initialize metrics on startup to fix restart gap; ProbeTx can be disabled#6
KA-ROM wants to merge 8 commits intomainfrom
metric_init_with_0

Conversation

@KA-ROM
Copy link

@KA-ROM KA-ROM commented Feb 10, 2026

Overview

  • Added Enabled bool to ProbeTx config — probes now require both enabled: true and a private key to activate
  • Init some metrics to 0 at startup for all known label combinations.

Fixes

  • Added missing stream_type label to flashblocks_receive_failure_count

Why metric changes

After service restarts, metrics disappear until their first events occur — creating gaps in observability and preventing alerts from firing despite chain-monitor being healthy. This PR fixes that by initializing all synchronous metrics to 0 at startup for all known label combinations.

New metric

  • FlashblocksStreamUp gauge to track stream connectivity (1=up, 0=down)

Metric zero-initialization at startup

  • FlashblocksMonitor.initializeMetricsFlashblocksStreams(): initializes per-stream metrics for all configured private/public streams:
    • flashblocks_stream_up
    • flashblocks_receive_success_count
    • flashblocks_receive_failure_count
  • BlockInspector.initializeMetrics(): initializes block inspector metrics:
    • blocks_seen_count, blocks_landed_count, blocks_missed_count, block_missed (always)
    • flashblocks_landed_count, flashblocks_missed_count (when flashblock number contract configured)
    • flashtestations_landed_count, flashtestations_missed_count, workload_added_to_policy_error_count (when builder policy contract configured)
    • registered_flashtestations_error_count (when flashtestation registry contract configured)

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Ensures flashblocks stream metrics are present immediately after startup/restart by initializing metrics for all configured streams and adding a stream connectivity gauge.

Changes:

  • Initialize flashblocks stream metrics to 0 for all configured streams during startup.
  • Add flashblocks_stream_up gauge and record 1/0 on successful connect / failures.
  • Standardize metric labels across flashblocks stream metrics (e.g., stream_type, network_id).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File Description
server/l2/flashblocks_monitor.go Initializes per-stream metrics on startup; adds stream_type labeling and records flashblocks_stream_up on connect/failure paths.
metrics/metrics.go Registers the new flashblocks_stream_up gauge instrument.
metrics/exports.go Exports the new gauge and wires its setup into metric initialization.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 100 to +109
processingContext, cancel := context.WithCancel(processingContext)
fm.stop = cancel

fm.initializeMetricsFlashblocksStreams(ctx)

for stream, url := range fm.cfg.privateStreams {
fm.readStream(ctx, stream, url, fm.flashblocksPrivate)
fm.readStream(ctx, stream, url, "private", fm.flashblocksPrivate)
}
for stream, url := range fm.cfg.publicStreams {
fm.readStream(ctx, stream, url, fm.flashblocksPublic)
fm.readStream(ctx, stream, url, "public", fm.flashblocksPublic)
Copy link

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code sets fm.stop to cancel processingContext, but initializeMetricsFlashblocksStreams and the stream goroutines are started with ctx instead of processingContext. If fm.stop() is called, processingContext will be canceled but the streams may keep running (and continue recording metrics) because they’re not observing the canceled context. Pass processingContext to initializeMetricsFlashblocksStreams(...) and to readStream(...) so shutdown behaves as intended.

Copilot uses AI. Check for mistakes.
@KA-ROM KA-ROM marked this pull request as draft February 10, 2026 11:41
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@KA-ROM KA-ROM marked this pull request as ready for review February 10, 2026 11:47
@KA-ROM KA-ROM changed the title Initialize flashblocks stream metrics on startup to fix restart gap fix: Initialize flashblocks stream metrics on startup to fix restart gap Feb 10, 2026
@KA-ROM KA-ROM changed the title fix: Initialize flashblocks stream metrics on startup to fix restart gap fix: Initialize metrics on startup to fix restart gap; ProbeTx can be disabled Feb 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants