Skip to content

Conversation

@RodrigoVillar
Copy link
Contributor

@RodrigoVillar RodrigoVillar commented Dec 2, 2025

Why this should be merged

As part of #4508, this PR removes usage of go bench from the reexecution test. This offers the following benefits:

  • Although the reexecution test is housed within a go benchmark, we only run it once, which is not idiomatic as go benchmarks are meant to be iterated over multiple times.
  • This sets up the foundation to reintroduce multiple metrics (e.g. block acceptance time, block verification time, etc.) to our dashboard as adding additional metrics is as simple as calling addResult() on the new benchmark tool.

How this works

  • Removes usage of *testing.B
  • Output of the reexecution statistics is handled by the test itself rather than ./scripts/benchmark_cchain_range.sh
  • Updates GAB (github-action-benchmark) action to use custom benchmarks rather than go bench

How this was tested

CI + ran task reexecute-cchain-range START_BLOCK=1 END_BLOCK=100_000 BENCHMARK_OUTPUT_FILE=<INSERT_CUSTOM_PATH_HERE> locally and got the following:

[
  {
    "name": "BenchmarkReexecuteRange/[1,100000]-Config-default-Runner-dev",
    "value": "254.90228109633716",
    "unit": "mgas/s"
  }
]

To test that benchmark results display correctly on the GAB dashboard, I created a sample benchmark (https://github.com/RodrigoVillar/sample-reexecution-custom-benchmark/blob/main/main.go) which is a copy-and-paste of the benchmarking code in this PR and plotted some points into my own dashboard. The results of the sample benchmark can be seen below:

Screenshot 2025-12-04 at 09 35 04

Attached is a link to the sample benchmark dashboard: https://rodrigovillar.github.io/sample-reexecution-custom-benchmark/dev/bench/

Need to be documented in RELEASES.md?

No

TODOs after merging

  • Prune gh-pages branch of data points which use go bench

@RodrigoVillar RodrigoVillar force-pushed the rodrigo/custom-benchmarks branch from bd29254 to 1935adf Compare December 3, 2025 23:36
@RodrigoVillar RodrigoVillar self-assigned this Dec 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants