Skip to content

feat: chain orchestrator #185

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 39 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
c1e5cc2
feat: chain orchestrator
frisitano Jul 2, 2025
bbe0e8f
Merge branch 'main' into feat/chain-orchestrator
frisitano Jul 2, 2025
4032fa5
lint
frisitano Jul 2, 2025
39802e2
lint
frisitano Jul 2, 2025
c84dc0d
docs lint
frisitano Jul 2, 2025
ea3140c
fix: clone the in-memory chain instead of taking it
frisitano Jul 3, 2025
a28a5b2
Merge branch 'main' into feat/chain-orchestrator
frisitano Jul 3, 2025
fa6952c
update chain consolidation and l1 message validation
frisitano Jul 4, 2025
43cc2be
add missing network block error
frisitano Jul 7, 2025
5dfc813
update fork sync / reconcilliation to use network as opossed to database
frisitano Jul 8, 2025
a50558d
Merge branch 'main' into feat/chain-orchestrator
frisitano Jul 10, 2025
19458f7
fix merege
frisitano Jul 10, 2025
4578d49
refactor and add test cases
frisitano Jul 16, 2025
e6248f5
cleanup
frisitano Jul 16, 2025
626e29b
Merge branch 'main' into feat/chain-orchestrator
frisitano Jul 16, 2025
7a6ebfc
rename chain orchestrator crate
frisitano Jul 16, 2025
7d35185
cleanup
frisitano Jul 17, 2025
74fd7b8
Merge branch 'main' into feat/chain-orchestrator
frisitano Jul 18, 2025
fe75ed5
remove expect for sequencer in rnm
frisitano Jul 21, 2025
f3bab0d
sync test
frisitano Jul 21, 2025
56a0ff3
sync test
frisitano Jul 21, 2025
1320f14
sync test
frisitano Jul 21, 2025
0fa87fe
add error handling for missing paylaod id
frisitano Jul 21, 2025
cecbd03
remove networking for L1 consolidation sync test
frisitano Jul 21, 2025
af4bac9
Merge branch 'main' into feat/chain-orchestrator
frisitano Jul 29, 2025
7df1a59
improve test coverage and fix bugs
frisitano Jul 30, 2025
fa51172
lint
frisitano Jul 30, 2025
e86446d
fix cli NetworkArgs
frisitano Jul 30, 2025
9cdbdf3
add block gap to reorg integration test
frisitano Jul 31, 2025
01a10cf
Merge branch 'main' into feat/chain-orchestrator
frisitano Jul 31, 2025
2f851b1
make test more robust
frisitano Jul 31, 2025
9f88b5d
Merge branch 'main' into feat/chain-orchestrator
frisitano Aug 6, 2025
9a3339a
merge upstream
frisitano Aug 7, 2025
000ade7
refactor
frisitano Aug 7, 2025
00d18f2
address comments
frisitano Aug 11, 2025
2beb7e0
Merge branch 'main' into feat/chain-orchestrator
frisitano Aug 11, 2025
add9933
address comments
frisitano Aug 20, 2025
e3d5453
Merge branch 'main' into feat/chain-orchestrator
frisitano Aug 20, 2025
4383231
merge changes
frisitano Aug 20, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .codespellrc
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[codespell]
skip = .git,target,Cargo.toml,Cargo.lock
skip = .git,target,Cargo.toml,Cargo.lock,docker-compose
ignore-words-list = crate
4 changes: 2 additions & 2 deletions .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -128,11 +128,11 @@ jobs:
- type: wasm
target: wasm32-unknown-unknown
exclude: |
scroll-engine,scroll-wire,rollup-node,scroll-network,rollup-node-manager,rollup-node-watcher,scroll-db,scroll-migration,rollup-node-indexer,scroll-codec,scroll-derivation-pipeline,rollup-node-providers,rollup-node-sequencer,rollup-node-signer,tests
scroll-engine,scroll-wire,rollup-node,scroll-network,rollup-node-manager,rollup-node-watcher,scroll-db,scroll-migration,rollup-node-chain-orchestrator,scroll-codec,scroll-derivation-pipeline,rollup-node-providers,rollup-node-sequencer,rollup-node-signer,tests
- type: riscv
target: riscv32imac-unknown-none-elf
exclude: |
scroll-engine,scroll-wire,rollup-node,scroll-network,rollup-node-manager,rollup-node-watcher,scroll-db,scroll-migration,rollup-node-indexer,scroll-codec,scroll-derivation-pipeline,rollup-node-providers,rollup-node-sequencer,rollup-node-signer,tests
scroll-engine,scroll-wire,rollup-node,scroll-network,rollup-node-manager,rollup-node-watcher,scroll-db,scroll-migration,rollup-node-chain-orchestrator,scroll-codec,scroll-derivation-pipeline,rollup-node-providers,rollup-node-sequencer,rollup-node-signer,tests
steps:
- uses: actions/checkout@v5
- uses: rui314/setup-mold@v1
Expand Down
27 changes: 25 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ members = [
"crates/database/migration",
"crates/derivation-pipeline",
"crates/engine",
"crates/indexer",
"crates/chain-orchestrator",
"crates/l1",
"crates/manager",
"crates/network",
Expand Down Expand Up @@ -155,6 +155,7 @@ reth-primitives-traits = { git = "https://github.com/scroll-tech/reth.git", defa
reth-provider = { git = "https://github.com/scroll-tech/reth.git", default-features = false }
reth-rpc-builder = { git = "https://github.com/scroll-tech/reth.git", default-features = false }
reth-rpc-server-types = { git = "https://github.com/scroll-tech/reth.git", default-features = false }
reth-storage-api = { git = "https://github.com/scroll-tech/reth.git", default-features = false }
reth-tasks = { git = "https://github.com/scroll-tech/reth.git", default-features = false }
reth-tokio-util = { git = "https://github.com/scroll-tech/reth.git", default-features = false }
reth-tracing = { git = "https://github.com/scroll-tech/reth.git", default-features = false }
Expand All @@ -168,7 +169,7 @@ reth-scroll-primitives = { git = "https://github.com/scroll-tech/reth.git", defa

# rollup node
rollup-node = { path = "crates/node" }
rollup-node-indexer = { path = "crates/indexer" }
rollup-node-chain-orchestrator = { path = "crates/chain-orchestrator" }
rollup-node-manager = { path = "crates/manager" }
rollup-node-primitives = { path = "crates/primitives" }
rollup-node-providers = { path = "crates/providers" }
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ This repository is a modular Rust workspace for the Scroll rollup node. It is de
│ │ └── migration/
│ ├── derivation-pipeline/
│ ├── engine/
│ ├── indexer/
│ ├── chain-orchestrator/
│ ├── l1/
│ ├── network/
│ ├── node/
Expand All @@ -46,7 +46,7 @@ This repository is a modular Rust workspace for the Scroll rollup node. It is de
- **crates/database/migration/**: Database schema migrations using SeaORM.
- **crates/derivation-pipeline/**: Stateless pipeline for transforming batches into block-building payloads.
- **crates/engine/**: Core engine logic for block execution, fork choice, and payload management.
- **crates/indexer/**: Indexes L1 and L2 data for efficient querying and notification.
- **crates/chain-orchestrator/**: Responsible for orchestrating the L2 chain based on events from L1 and data gossiped over the P2P network.
- **crates/l1/**: Primitives and ABI bindings for L1 contracts and messages.
- **crates/network/**: P2P networking stack for node communication.
- **crates/node/**: Node manager and orchestration logic.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[package]
name = "rollup-node-indexer"
name = "rollup-node-chain-orchestrator"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
Expand All @@ -11,30 +11,45 @@ workspace = true

[dependencies]
# alloy
alloy-consensus = { workspace = true }
alloy-eips = { workspace = true }
alloy-json-rpc.workspace = true
alloy-primitives.workspace = true
alloy-provider.workspace = true
alloy-transport.workspace = true

# rollup-node
scroll-db.workspace = true
rollup-node-primitives.workspace = true
rollup-node-watcher.workspace = true

# scroll
reth-scroll-primitives.workspace = true
scroll-alloy-consensus.workspace = true
scroll-alloy-hardforks.workspace = true
scroll-alloy-network.workspace = true
scroll-network.workspace = true

# reth
reth-chainspec.workspace = true
reth-network-p2p = { git = "https://github.com/scroll-tech/reth.git", default-features = false }
reth-network-peers.workspace = true
reth-primitives-traits.workspace = true

# misc
futures.workspace = true
metrics.workspace = true
metrics-derive.workspace = true
strum = "0.27.1"
thiserror.workspace = true
tracing.workspace = true
tokio.workspace = true

[dev-dependencies]
alloy-consensus = { workspace = true, features = ["arbitrary"] }
alloy-primitives = { workspace = true, features = ["arbitrary"] }
alloy-rpc-client.workspace = true
alloy-transport.workspace = true

# rollup-node
scroll-db = { workspace = true, features = ["test-utils"] }
Expand All @@ -44,8 +59,15 @@ rollup-node-primitives = { workspace = true, features = ["arbitrary"] }
reth-scroll-chainspec.workspace = true
reth-scroll-forks.workspace = true

# reth
reth-eth-wire-types.workspace = true
reth-network-peers.workspace = true

# misc
arbitrary.workspace = true
futures.workspace = true
parking_lot.workspace = true
rand.workspace = true
reqwest.workspace = true
serde_json = { version = "1.0" }
tokio.workspace = true
57 changes: 57 additions & 0 deletions crates/chain-orchestrator/src/action.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
use super::{ChainOrchestratorError, ChainOrchestratorEvent};
use std::{
fmt,
future::Future,
pin::Pin,
task::{Context, Poll},
};

/// A future that resolves to a `Result<ChainOrchestratorEvent, ChainOrchestratorError>`.
pub(super) type PendingChainOrchestratorFuture = Pin<
Box<dyn Future<Output = Result<Option<ChainOrchestratorEvent>, ChainOrchestratorError>> + Send>,
>;

/// A type that represents a future that is being executed by the chain orchestrator.
pub(super) enum ChainOrchestratorFuture {
HandleReorg(PendingChainOrchestratorFuture),
HandleFinalized(PendingChainOrchestratorFuture),
HandleBatchCommit(PendingChainOrchestratorFuture),
HandleBatchFinalization(PendingChainOrchestratorFuture),
HandleL1Message(PendingChainOrchestratorFuture),
HandleDerivedBlock(PendingChainOrchestratorFuture),
HandleL2Block(PendingChainOrchestratorFuture),
}

impl ChainOrchestratorFuture {
/// Polls the future to completion.
pub(super) fn poll(
&mut self,
cx: &mut Context<'_>,
) -> Poll<Result<Option<ChainOrchestratorEvent>, ChainOrchestratorError>> {
match self {
Self::HandleReorg(fut) |
Self::HandleFinalized(fut) |
Self::HandleBatchCommit(fut) |
Self::HandleBatchFinalization(fut) |
Self::HandleL1Message(fut) |
Self::HandleDerivedBlock(fut) |
Self::HandleL2Block(fut) => fut.as_mut().poll(cx),
}
}
}

// We implement the Debug trait for ChainOrchestratorFuture to provide a human-readable
// representation of the enum variants.
impl fmt::Debug for ChainOrchestratorFuture {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::HandleReorg(_) => write!(f, "HandleReorg"),
Self::HandleFinalized(_) => write!(f, "HandleFinalized"),
Self::HandleBatchCommit(_) => write!(f, "HandleBatchCommit"),
Self::HandleBatchFinalization(_) => write!(f, "HandleBatchFinalization"),
Self::HandleL1Message(_) => write!(f, "HandleL1Message"),
Self::HandleDerivedBlock(_) => write!(f, "HandleDerivedBlock"),
Self::HandleL2Block(_) => write!(f, "HandleL2Block"),
}
}
}
47 changes: 47 additions & 0 deletions crates/chain-orchestrator/src/error.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
use alloy_json_rpc::RpcError;
use alloy_primitives::B256;
use alloy_transport::TransportErrorKind;
use scroll_db::{DatabaseError, L1MessageStart};

/// A type that represents an error that occurred in the chain orchestrator.
#[derive(Debug, thiserror::Error)]
pub enum ChainOrchestratorError {
/// An error occurred while interacting with the database.
#[error("database error occurred: {0}")]
DatabaseError(#[from] DatabaseError),
/// An error occurred while trying to fetch the L2 block from the database.
#[error("L2 block not found - block number: {0}")]
L2BlockNotFoundInDatabase(u64),
/// An error occurred while trying to fetch the L2 block from the L2 client.
#[error("L2 block not found in L2 client - block number: {0}")]
L2BlockNotFoundInL2Client(u64),
/// A fork was received from the peer that is associated with a reorg of the safe chain.
#[error("L2 safe block reorg detected")]
L2SafeBlockReorgDetected,
/// A block contains invalid L1 messages.
#[error("Block contains invalid L1 message. Expected: {expected:?}, Actual: {actual:?}")]
L1MessageMismatch {
/// The expected L1 messages hash.
expected: B256,
/// The actual L1 messages hash.
actual: B256,
},
/// An L1 message was not found in the database.
#[error("L1 message not found at {0}")]
L1MessageNotFound(L1MessageStart),
/// An inconsistency was detected when trying to consolidate the chain.
#[error("Chain inconsistency detected")]
ChainInconsistency,
/// The peer did not provide the requested block header.
#[error("A peer did not provide the requested block header")]
MissingBlockHeader {
/// The hash of the block header that was requested.
hash: B256,
},
/// An error occurred while making a network request.
#[error("Network request error: {0}")]
NetworkRequestError(#[from] reth_network_p2p::error::RequestError),
/// An error occurred while making a JSON-RPC request to the Execution Node (EN).
#[error("An error occurred while making a JSON-RPC request to the EN: {0}")]
RpcError(#[from] RpcError<TransportErrorKind>),
}
66 changes: 66 additions & 0 deletions crates/chain-orchestrator/src/event.rs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naming on the events seems a bit inconsistent after just reading the event names and description in this file.

Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
use alloy_consensus::Header;
use alloy_primitives::{Signature, B256};
use reth_network_peers::PeerId;
use reth_scroll_primitives::ScrollBlock;
use rollup_node_primitives::{BatchInfo, BlockInfo, ChainImport, L2BlockInfoWithL1Messages};

/// An event emitted by the `ChainOrchestrator`.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ChainOrchestratorEvent {
/// A new block has been received from the network but we have insufficient data to process it
/// due to being in optimistic mode.
InsufficientDataForReceivedBlock(B256),
/// The block that we have received is already known.
BlockAlreadyKnown(B256, PeerId),
/// A fork of the chain that is older than the current chain has been received.
OldForkReceived {
/// The headers of the old fork.
headers: Vec<Header>,
/// The peer that provided the old fork.
peer_id: PeerId,
/// The signature of the old fork.
signature: Signature,
},
/// The chain should be optimistically synced to the provided block.
OptimisticSync(ScrollBlock),
/// The chain has been extended, returning the new blocks.
ChainExtended(ChainImport),
/// The chain has reorged, returning the new chain and the peer that provided them.
ChainReorged(ChainImport),
/// A `BatchCommit` event has been indexed returning the batch info and the L2 block info to
/// revert to due to a batch revert.
Comment on lines +30 to +31
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this comment

BatchCommitIndexed {
/// The batch info.
batch_info: BatchInfo,
/// The L1 block number in which the batch was committed.
l1_block_number: u64,
/// The safe L2 block info.
safe_head: Option<BlockInfo>,
},
/// A batch has been finalized returning the batch hash and new an optional finalized
/// L2 block.
BatchFinalized(B256, Option<BlockInfo>),
/// An L1 block has been finalized returning the L1 block number and an optional
/// finalized L2 block.
L1BlockFinalized(u64, Option<BlockInfo>),
/// A `L1Message` event has been committed returning the message queue index.
L1MessageCommitted(u64),
/// The chain has been unwound, returning the L1 block number of the new L1 head,
/// the L1 message queue index of the new L1 head, and optionally the L2 head and safe block
/// info if the unwind resulted in a new L2 head or safe block.
ChainUnwound {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what does this mean? Is it a reorg?

/// The L1 block number of the new L1 head.
l1_block_number: u64,
/// The L1 message queue index of the new L1 head.
queue_index: Option<u64>,
/// The L2 head block info.
l2_head_block_info: Option<BlockInfo>,
/// The L2 safe block info.
l2_safe_block_info: Option<BlockInfo>,
},
/// An L2 block has been committed returning the [`L2BlockInfoWithL1Messages`] and an
/// optional [`BatchInfo`] if the block is associated with a committed batch.
L2ChainCommitted(L2BlockInfoWithL1Messages, Option<BatchInfo>, bool),
/// An L2 consolidated block has been committed returning the [`L2BlockInfoWithL1Messages`].
L2ConsolidatedBlockCommitted(L2BlockInfoWithL1Messages),
}
Loading