1/n feat: add preconfirmation support#3005
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## preconf-dev #3005 +/- ##
===============================================
- Coverage 62.69% 61.50% -1.19%
===============================================
Files 357 366 +9
Lines 17226 17862 +636
===============================================
+ Hits 10799 10986 +187
- Misses 5572 5998 +426
- Partials 855 878 +23
🚀 New features to boost your workflow:
|
fridrik01
changed the title
fridrik01
force-pushed
the
add-preconf-support
branch
from
48fad0a to
89cd61e
Compare
fridrik01
force-pushed
the
add-preconf-support
branch
from
89cd61e to
92b8c7f
Compare
fridrik01
changed the title
There was a problem hiding this comment.
Pull request overview
Implements beacon-kit support for a preconfirmation flow, adding a sequencer-side HTTP payload server plus validator-side fetching/fallback, and wiring this through config/DI and Kurtosis/e2e scaffolding.
Changes:
- Add
beacon/preconfmodule (config, whitelist/JWT loading, HTTP server, HTTP client) and integrate it into validator and blockchain optimistic-building logic. - Extend configuration + CLI flags + templates to enable preconf modes (sequencer vs validator fetching).
- Add Kurtosis + e2e support for spinning up a preconf devnet and validating basic behavior/fallback.
Reviewed changes
Copilot reviewed 45 out of 45 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
| testing/simulated/components.go | Includes preconf providers in simulated test component wiring. |
| testing/e2e/suite/suite.go | Adds helper to remove a Kurtosis service via Starlark to simulate outages. |
| testing/e2e/suite/options.go | Adds suite option to run e2e with preconf-enabled config. |
| testing/e2e/suite/logs.go | Adds Kurtosis service log collection helpers for assertions. |
| testing/e2e/e2e_preconf_test.go | Adds an e2e_preconf test validating serving/fetching and fallback behavior. |
| testing/e2e/config/defaults.go | Adds PreconfE2ETestConfig for preconf-enabled e2e runs. |
| testing/e2e/config/config.go | Extends e2e config schema with PreconfConfig. |
| scripts/build/testing.mk | Adds make test-e2e-preconf targets. |
| payload/builder/payload.go | Adds PayloadBuilder.GetPayloadBySlot for serving cached payloads to validators. |
| payload/builder/interfaces.go | Renames cache interface arg names to reflect “blockRoot” semantics. |
| node-core/components/validator_service.go | Injects preconf client/whitelist into validator service. |
| node-core/components/service_registry.go | Registers preconf server (sequencer mode) in service registry. |
| node-core/components/preconf_server.go | Depinject provider for preconf HTTP server. |
| node-core/components/preconf_client.go | Depinject provider for preconf HTTP client. |
| node-core/components/preconf.go | Depinject provider for loading/constructing the preconf whitelist. |
| node-core/components/chain_service.go | Injects preconf config/whitelist into blockchain service. |
| kurtosis/src/services/sequencer/launcher.star | Adds launcher for a dedicated sequencer EL service (bera-sequencer). |
| kurtosis/src/services/flashblock-monitor/launcher.star | Adds a simple websocat-based flashblock monitor service. |
| kurtosis/src/preconf/config.star | Generates whitelist/JWT artifacts and preconf CLI flags for devnet. |
| kurtosis/src/nodes/nodes.star | Adds parsing support for a dedicated sequencer node. |
| kurtosis/src/nodes/execution/reth/config.star | Adds reth sequencer-mode args + flashblock WS port wiring. |
| kurtosis/src/nodes/execution/execution.star | Enables sequencer-mode configuration for dedicated sequencer node. |
| kurtosis/src/nodes/consensus/beacond/node.star | Adds support for appending preconf flags to beacond start command. |
| kurtosis/src/nodes/consensus/beacond/launcher.star | Wires preconf artifacts/env into CL node configs; adjusts genesis ceremony return shape. |
| kurtosis/main.star | Adds preconf args handling; deploys dedicated sequencer node when enabled; launches related services. |
| kurtosis/beaconkit-preconf.yaml | Adds Kurtosis args file to run a preconf-enabled devnet (with sequencer_node). |
| kurtosis/Makefile | Adds targets to start/stop/remove a preconf devnet. |
| config/template/template.go | Adds TOML template section for [beacon-kit.preconf]. |
| config/config.go | Adds preconf config to global node config defaults and struct. |
| cmd/beacond/defaults.go | Adds preconf providers to default component list. |
| cli/flags/flags.go | Adds CLI flags for preconf config. |
| beacon/validator/service.go | Extends validator service to hold preconf cfg/client/whitelist. |
| beacon/validator/block_builder.go | Attempts sequencer fetch before local build when configured/whitelisted. |
| beacon/preconf/whitelist.go | Adds whitelist interface + implementation. |
| beacon/preconf/types.go | Defines request/response types and envelope conversion helpers. |
| beacon/preconf/server_test.go | Adds unit tests for server auth/method handling. |
| beacon/preconf/server.go | Implements JWT-authenticated preconf payload server endpoint. |
| beacon/preconf/loader_test.go | Adds unit tests for whitelist/JWT loader helpers. |
| beacon/preconf/loader.go | Implements whitelist/JWT secret file loaders. |
| beacon/preconf/config.go | Adds preconf config struct + defaults + helper predicates. |
| beacon/preconf/client.go | Implements HTTP client for fetching payloads from sequencer. |
| beacon/blockchain/service.go | Adds preconf cfg/whitelist + optimistic-build trigger tracking to blockchain service. |
| beacon/blockchain/process_proposal.go | Adds sequencer-mode proposer detection + optimistic build triggering based on whitelist. |
| beacon/blockchain/payload_test.go | Updates blockchain tests to pass new constructor args. |
| beacon/blockchain/finalize_block.go | Adds FinalizeBlock fallback to trigger optimistic build when ProcessProposal is skipped. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if s.preconfCfg.IsSequencer() && s.localBuilder.Enabled() { | ||
| skipped := !s.optimisticBuildTriggered.Load() | ||
| s.optimisticBuildTriggered.Store(false) | ||
| if skipped { | ||
| s.sequencerFinalizeOptimisticBuild(ctx, blk, math.U64(req.GetTime().Unix())) //#nosec: G115 |
There was a problem hiding this comment.
s.preconfCfg is used without a nil check (s.preconfCfg.IsSequencer()), but blockchain.NewService accepts a potentially nil *preconf.Config. This can panic in FinalizeBlock (and tests currently construct the service with nil). Either ensure preconfCfg is always non-nil (default config) or guard this block with s.preconfCfg != nil.
There was a problem hiding this comment.
just sanity checking since I dont see where *preconf.Config is initialized and passed in to NewService. Do we ensure its non-nil? Or maybe making the type a non-pointer so nil issue goes away?
fridrik01
force-pushed
the
add-preconf-support
branch
from
43712f1 to
fb4a434
Compare
fridrik01
force-pushed
the
add-preconf-support
branch
from
fb4a434 to
354d20d
Compare
fridrik01
force-pushed
the
add-preconf-support
branch
from
354d20d to
4cdc1dd
Compare
…d kurtosis devnet improvements
fridrik01
force-pushed
the
add-preconf-support
branch
from
e5f5da4 to
455ffc7
Compare
fridrik01
force-pushed
the
add-preconf-support
branch
from
8794989 to
3e32cca
Compare
bar-bera
left a comment
bar-bera
left a comment
There was a problem hiding this comment.
A couple of observations:
-
description says
Validator Mode (whitelisted):
- ...
- Validates the payload by calling newPayload on the local execution client
Seems to me validation of fetched blocks from the sequencer is still missing. Is this something we want to address on a dedicated PR?
-
minor simplification: non-sequencer validators use whitelist to see if they
shouldFetchFromSequencer(). since the whitelist is just a local file this seems a bit weird, maybe just checking if they are configured to use preconf is enough to assume they are already whitelisted? who would set the preconf path without being whitelisted ahead? or maybe differentiating the config path for sequencer and non-sequencer creates more confusion than other.
Good catch, yes I had started implemented this locally but removed it since the PR was already so big but forgot to update the PR summary (fixed now). I have created a new clickup task to track the progress on validating sequencer payload.
You are right and I noticed this when I deployed preconfs to our kubernetes infra, I fixed it in #3054 |
calbera
left a comment
calbera
left a comment
There was a problem hiding this comment.
some nits mainly, but overall LGTM and great work! love to see kurtosis being used/useful :p
| if s.preconfCfg.IsSequencer() && s.localBuilder.Enabled() { | ||
| skipped := !s.optimisticBuildTriggered.Load() | ||
| s.optimisticBuildTriggered.Store(false) | ||
| if skipped { | ||
| s.sequencerFinalizeOptimisticBuild(ctx, blk, math.U64(req.GetTime().Unix())) //#nosec: G115 |
There was a problem hiding this comment.
just sanity checking since I dont see where *preconf.Config is initialized and passed in to NewService. Do we ensure its non-nil? Or maybe making the type a non-pointer so nil issue goes away?
| return | ||
| } | ||
|
|
||
| s.handleOptimisticPayloadBuild(ctx, buildData) |
There was a problem hiding this comment.
do we want to do this in a go routine as we trigger in process proposal (standard path)? or is it important to wait because of finalizeBLock; just want to make sure we don't exceed any comet finalize block timeouts
| // We're the sequencer, build for whitelisted validators. | ||
| // Only check whitelist if we can resolve the next proposer pubkey. |
There was a problem hiding this comment.
question: since we're sequencer, we "optimistically" build in finalize block even if the next proposer is not part of whitelist. Why couldn't we just remove this check here in process proposal and always have sequencer trigger optimistic build in process proposal? finalize block changes made in this PR would not be necessary then
| } | ||
|
|
||
| // ErrorResponse is the error response body. | ||
| type ErrorResponse struct { |
There was a problem hiding this comment.
might be dumb, but should this also implement the http error interface to get the right HTTP errors? was smth like ErrorCode() and 1 other
| Get(slot math.Slot, stateRoot common.Root) (cache.PayloadIDCacheResult, bool) | ||
| Set(slot math.Slot, stateRoot common.Root, pid engineprimitives.PayloadID, version common.Version) | ||
| Delete(slot math.Slot, stateRoot common.Root) | ||
| Get(slot math.Slot, blockRoot common.Root) (cache.PayloadIDCacheResult, bool) |
There was a problem hiding this comment.
these are all parentBlockRoot correct?
fridrik01
changed the title
Context
This PR implements the beacon-kit portion of the preconfirmation system which significantly reduces transaction inclusion latency for users who opt into the preconfirmation network.
Berachain's standard block time is ~2 seconds. The preconfirmation system provides a fast path where a sequencer emits partial blocks (flashblocks) every ~200ms (not implemented in this PR), which are then finalized when the full block is proposed through CometBFT consensus. If any component fails, the chain seamlessly falls back to normal block building.
Design
The preconfirmation network consists of a sequencer and a set of whitelisted validators. Each runs beacon-kit with different roles:
Sequencer Mode:
N, looks up the proposer for slotN+1forkchoiceUpdateto begin payload construction early/eth/v1/preconf/payload) where validators fetch their pre-built payloadsValidator Mode (whitelisted):
Validator Mode (not in whitelist):
Robustness: Race Condition Handling
The optimistic payload build is triggered from
ProcessProposalin a background goroutine. This introduces two race conditions that are handled by:GetPayloadBySlotretries up to 5 times (50ms intervals, 250ms max) onErrPayloadIDNotFound(cache miss) orErrUnknownPayload(EL still building).ProcessProposalentirely if a proposal arrives after the propose timeout (the block goes directly toFinalizeBlock). AnoptimisticBuildTriggeredatomic flag tracks whetherProcessProposaltriggered a build. IfFinalizeBlockfinds the flag unset,sequencerFinalizeOptimisticBuildtriggers the build as a fallback.Kurtosis Devnet
The kurtosis devnet config (kurtosis/beaconkit-preconf.yaml) sets up a multi-node network with a dedicated sequencer:
--sequencer-enabled, flashblock WebSocket (port8548), and a signing key--flashblocks-urlto consume the sequencer's flashblock stream. This node serves standard JSON-RPC with. flashblock-aware stateTest Plan
Checkout branch
sequencer-devfrom feat: add sequencer/rpc mode for flashblock preconfirmations bera-reth#214 locally and runmake docker-build-localin the bera-reth directoryRun
make start-devnet-preconfto start local kurtosis devnetView
flashblock-monitorcontainer logs and you should see flashblocks produced every 200msOutstanding Issues
sequencerFinalizeOptimisticBuildfallback: I haven't been able to get this to happen locally after introducing this fallback, though it happened a few times before.