types: migrate Gloas + consensus-data SSZ to pk910/dynamic-ssz - #643
types: migrate Gloas + consensus-data SSZ to pk910/dynamic-ssz#643iurii-ssv wants to merge 5 commits into
Conversation
Greptile SummaryThe PR migrates Gloas and consensus-data SSZ encoding and hashing from fastssz to dynamic-ssz, updates the associated beacon types and signing interface, and changes supporting dependencies.
Confidence Score: 4/5The runtime migration appears coherent, but the PR should not merge until consensus-data generation uses only the new dynamic-ssz path. Running the repository’s generation command still invokes fastssz for the migrated consensus-data structs before dynssz-gen generates the same SSZ method set, making checked-in encoder regeneration fail or produce conflicting declarations. Files Needing Attention: types/generate.go
|
| Filename | Overview |
|---|---|
| types/generate.go | Adds dynssz-gen for consensus-data but retains the overlapping legacy fastssz generator, breaking regeneration. |
| types/consensus_data.go | Updates consensus-data types and bridges composite-list hashing through each dynamic type’s canonical root. |
| types/ssz.go | Introduces the narrow HashRoot interface and adapts withdrawals-list hashing for dynamic-ssz elements. |
| types/gloas/beaconblockbody_ssz.go | Provides generated dynamic-ssz encoding and progressive merkleization for the Gloas block body. |
| ssv/proposer.go | Routes Gloas proposer blocks through the new decoder and HashRoot signing path. |
| go.mod | Raises the Go version, adds dynamic-ssz tooling and dependencies, changes bitfield implementation, and pins the preview go-eth2-client fork. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart LR
S[Consensus and Gloas source types] --> D[dynssz-gen]
D --> E[Dynamic SSZ encoders and roots]
E --> C[Consensus values]
C --> R[Duty runners]
R --> H[HashTreeRoot]
H --> P[Partial signatures and beacon submission]
S -. legacy overlapping path .-> F[fastssz sszgen]
F -. duplicate consensus-data methods .-> E
Comments Outside Diff (1)
-
types/generate.go, line 18-25 (link)Overlapping SSZ generators break regeneration
When
go generate ./typesruns, it first regenerates the migrated consensus-data methods with fastssz and then generates the same method set with dynssz-gen, causing generation failures or duplicate declarations in the package.Knowledge Base Used: Restore dependency and code-generation compatibility
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Reviews (1): Last reviewed commit: "types: migrate Gloas + consensus-data SS..." | Re-trigger Greptile
658d6b1 to
e6e01e5
Compare
e6e01e5 to
05dea41
Compare
maxmsgsize: build the phase0/electra max-attestation fixtures with valid
bitlists (bitfield.NewBitlist) so they encode under dynamic-ssz. A bitlist's
byte length can't equal its bit-count ssz-max, so these structure-size tests
now check ≤ and pin the exact size (485 / 16621) via ExpectedEncodedLength;
MaxSize{Phase0,Electra}Attestation are documented as intentional
over-approximations feeding the AggregatedAttestations list limit.
generate: resolve the go-eth2-client include via `go list -m` (wrapped in sh -c)
instead of the hard-coded $GOPATH/...@v0.27.0 path, which no longer exists under
the fork replace. Drop the no-op gloas `rm -f *_ssz.go` (go generate never
expanded the glob) and document that dynssz-gen overwrites the encoders in
place: it must type-check the package to load it, and the hand-written
Encode/Decode wrappers reference the generated MarshalSSZ, so the files can't be
deleted first — a stale encoder for a removed type fails the build instead.
gloas: note that ssv-spec keeps its own Gloas containers (spec-owned, pinned to
the fork by beacon_block_root_test.go) rather than aliasing the fork; align
ExecutionPayloadBid.PrevRandao with the fork (phase0.Root, same SSZ); fix a
devnet-6/devnet-8 comment.
4e04768 to
c719c7d
Compare
|
Re: the "Overlapping SSZ generators break regeneration" P1 — this was reviewed against an earlier commit and is already resolved at HEAD. |
maxmsgsize: build the phase0/electra max-attestation fixtures with valid bitlists (bitfield.NewBitlist) so they encode under dynamic-ssz. A bitlist's byte length can't equal its bit-count ssz-max, so these structure-size tests now check ≤ and pin the exact size (485 / 16621) via ExpectedEncodedLength. MaxSizeElectraAttestation (which bounds the AggregatedAttestations list element) is documented as an intentional over-approximation; the now-unused MaxSizePhase0Attestation is dropped. generate: resolve the go-eth2-client include via `go list -m` (wrapped in sh -c) instead of the hard-coded $GOPATH/...@v0.27.0 path, which no longer exists under the fork replace. Drop the no-op gloas `rm -f *_ssz.go` (go generate never expanded the glob) and document that dynssz-gen overwrites the encoders in place: it must type-check the package to load it, and the hand-written Encode/Decode wrappers reference the generated MarshalSSZ, so the files can't be deleted first — a stale encoder for a removed type fails the build instead. gloas: note that ssv-spec keeps its own Gloas containers (spec-owned, pinned to the fork by beacon_block_root_test.go) rather than aliasing the fork; align ExecutionPayloadBid.PrevRandao with the fork (phase0.Root, same SSZ); fix a devnet-6/devnet-8 comment.
c719c7d to
95a0124
Compare
… roots Aggregator duties failed on Gloas slots with "unknown data version: 8" (#3009): the fork already decodes the gloas Eth-Consensus-Version into VersionedAttestation.Gloas and ships a distinct gloas.AggregateAndProof (its Attestation merkleizes progressively, so its root differs from the byte-identical Electra container's), but the node's version switches had no Gloas arm. Add it to the goclient aggregate conversions, the consensus-data decode, and a signed-aggregate constructor now shared by both aggregator runners; the remote signer rejects Gloas aggregates explicitly (#3000). Drop ForkAtEpoch's Fulu cap in the node and ssvsigner: a scheduled Gloas fork resolves from its epoch on and absent forks are skipped, so attestations ride gloas.Attestation and the aggregator-committee consensus data stamps DataVersionGloas (SIP #94 §2, #2998). This needs ssv-spec's Gloas arms to build the Gloas containers, so both go.mods repin to the ssv-spec dynssz branch head that carries them (ssvlabs/ssv-spec#643). The remote signer's fork_info substitution is no longer needed. Pin the Gloas roots to the chain (#3008): a devnet-8 finalized block whose HashTreeRoot must equal the chain's header root and whose proposer signature must verify, plus its execution-payload envelope, which must blind to the full envelope's root. EIP-7688/7916 change only merkleization, so round-trip tests alone cannot catch a wrong root.
5461cb3 to
3b6adb7
Compare
maxmsgsize: build the phase0/electra max-attestation fixtures with valid bitlists (bitfield.NewBitlist) so they encode under dynamic-ssz. A bitlist's byte length can't equal its bit-count ssz-max, so these structure-size tests now check ≤ and pin the exact size (485 / 16621) via ExpectedEncodedLength. MaxSizeElectraAttestation (which bounds the AggregatedAttestations list element) is documented as an intentional over-approximation; the now-unused MaxSizePhase0Attestation is dropped. generate: resolve the go-eth2-client include via `go list -m` (wrapped in sh -c) instead of the hard-coded $GOPATH/...@v0.27.0 path, which no longer exists under the fork replace. Drop the no-op gloas `rm -f *_ssz.go` (go generate never expanded the glob) and document that dynssz-gen overwrites the encoders in place: it must type-check the package to load it, and the hand-written Encode/Decode wrappers reference the generated MarshalSSZ, so the files can't be deleted first — a stale encoder for a removed type fails the build instead. gloas: note that ssv-spec keeps its own Gloas containers (spec-owned, pinned to the fork by beacon_block_root_test.go) rather than aliasing the fork; align ExecutionPayloadBid.PrevRandao with the fork (phase0.Root, same SSZ); fix a devnet-6/devnet-8 comment.
b894983 to
f0772bc
Compare
Gloas (EIP-7688/7916) makes the affected containers/lists progressive, which fastssz cannot merkleize, so the hand-rolled fastssz encoders computed wrong hash_tree_roots. Move the Gloas types and the consensus-data types that embed go-eth2-client composite types onto pk910/dynamic-ssz (matching go-eth2-client's gloas branch) and reconcile the fastssz -> dynamic-ssz interface change: - types/gloas: regenerate encoders with dynssz-gen; port ssz-index + ssz-type:"progressive-list" tags. BeaconBlockBody now uses gloas.Attestation / gloas.AttesterSlashing (not electra). - types: regenerate the consensus-data encoders (BeaconVote, GloasBeaconVote, Contribution, ProposerConsensusData, ...) with dynssz-gen; the rest of the package stays on fastssz. types/generate.go generates the consensus-data types only via dynssz-gen (dropped the overlapping fastssz directive). - go-bitfield: prysmaticlabs -> OffchainLabs (matches go-eth2-client). - ssz.HashRoot -> a narrow HashRoot interface (dynamic-ssz types implement HashTreeRoot() but not fastssz's GetTree()). - Hand-written list helpers (SSZWithdrawals, Contributions) hash via each element's HashTreeRoot() instead of fastssz's HashTreeRootWith. - Add committed HTR-vs-go-eth2-client regression tests (types/gloas): a real devnet block (progressive container) and a non-empty progressive list, each decoded into both types and asserted equal — pinning progressive merkleization in CI, independent of the external spec-tests repo. - Refresh beacon_types_encoding.go (fastssz Hash reflects the resolved fork) so `go generate ./types/` is idempotent; document the generated (*_ssz.go) vs hand-written (snake_case.go) split in types/gloas/generate.go. - go.mod: go 1.25, + dynamic-ssz, + OffchainLabs/go-bitfield, dynssz-gen tool, temporary replace to ssvlabs/go-eth2-client@fork-gloas until gloas releases. - CI: bump lint + generate-spec-tests actions to Go 1.25.x for the go.mod floor and the tool directive. Stacked on epbs-gloas-types. BuilderDepositRequest keeps [32]byte WithdrawalCredentials (same root as go-eth2-client's []byte ssz-size:32).
BeaconVoteDataNil builds a BeaconVote with nil Source/Target, then passed *consensusData.Source / *consensusData.Target to NewSpecTest, which takes phase0.Checkpoint by value. That deref only worked by accident: fastssz's MarshalSSZ allocated the nil checkpoints on the receiver, so by the time the derefs ran Source/Target were non-nil zero checkpoints. This package's move to dynamic-ssz — whose MarshalSSZ does not mutate the receiver — leaves them nil, so the derefs panic and take down the whole vector generator (no ssv spec-test vectors could be produced). Pass zero checkpoints explicitly: a nil Source/Target SSZ-encodes to a zero checkpoint, so that is the value the decoded vote carries, and the test still asserts the source-not-less-than-target rejection.
maxmsgsize: build the phase0/electra max-attestation fixtures with valid bitlists (bitfield.NewBitlist) so they encode under dynamic-ssz. A bitlist's byte length can't equal its bit-count ssz-max, so these structure-size tests now check ≤ and pin the exact size (485 / 16621) via ExpectedEncodedLength. MaxSizeElectraAttestation (which bounds the AggregatedAttestations list element) is documented as an intentional over-approximation; the now-unused MaxSizePhase0Attestation is dropped. generate: resolve the go-eth2-client include via `go list -m` (wrapped in sh -c) instead of the hard-coded $GOPATH/...@v0.27.0 path, which no longer exists under the fork replace. Drop the no-op gloas `rm -f *_ssz.go` (go generate never expanded the glob) and document that dynssz-gen overwrites the encoders in place: it must type-check the package to load it, and the hand-written Encode/Decode wrappers reference the generated MarshalSSZ, so the files can't be deleted first — a stale encoder for a removed type fails the build instead. gloas: note that ssv-spec keeps its own Gloas containers (spec-owned, pinned to the fork by beacon_block_root_test.go) rather than aliasing the fork; align ExecutionPayloadBid.PrevRandao with the fork (phase0.Root, same SSZ); fix a devnet-6/devnet-8 comment.
…ient's Gloas containers go-eth2-client's Gloas types give VersionedAttestation and VersionedAggregateAndProof their own Gloas members, and gloas.Attestation merkleizes as a progressive container with a progressive-bitlist aggregation_bits (EIP-7688 / EIP-7916): the same bytes as the Electra container, a different hash tree root. The Gloas arms here reused the Electra container, so on Gloas slots the aggregate-and-proof signing root would not match a beacon node's (ssvlabs/ssv#3009). Switch every Gloas arm to the Gloas containers: AggregatorCommitteeConsensusData.Validate and GetAggregateAndProofs, GetAggregateAndProofHashRoot, the aggregator-committee signed constructor, the committee runner's attestation helpers (new ConstructGloasAttestationWithoutSignature), and the testing beacon node and aggregator fixtures. DataVersionGloas becomes an alias of upstream's spec.DataVersionGloas (same value). The aggregator-committee Gloas vectors change roots and signatures; consensus-data bytes do not.
Fold the §5 BuilderRequestAuth / SignedBuilderRequestAuth types into the pk910/dynamic-ssz generation, matching the rest of the Gloas package on this branch: drop the fastssz sszgen directive and its _encoding.go output, add the two types to generate.yaml, and regenerate the flatcase *_ssz.go encoders. The slot runner's expectedRequestAuthRootsAndDomain returns the narrow types.HashRoot (dynssz types implement HashTreeRoot but not fastssz's GetTree).
f0772bc to
f461f5b
Compare
Stacked on #633 (
epbs-gloas-types). Migrates the Gloas types — and the consensus-data types that embed go-eth2-client composite types — fromferranbt/fastssztopk910/dynamic-ssz.Why
Gloas (EIP-7688 + EIP-7916) makes the affected containers/lists progressive, which fastssz cannot merkleize — so the hand-rolled fastssz encoders computed the wrong
hash_tree_root. Correct progressive merkleization has to come from dynamic-ssz, matching go-eth2-client'sgloasbranch.What
types/gloas— encoders regenerated withdynssz-gen;ssz-index+ssz-type:"progressive-list"tags ported.BeaconBlockBodynow usesgloas.Attestation/gloas.AttesterSlashing(not electra).types— consensus-data encoders (BeaconVote,GloasBeaconVote,Contribution,ProposerConsensusData, …) regenerated withdynssz-gen; the rest of the package stays on fastssz.types/generate.gogenerates the consensus-data types only via dynssz-gen (the overlapping fastssz directive was dropped).prysmaticlabs→OffchainLabs(matches go-eth2-client).ssz.HashRoot→ a narrowHashRootinterface (dynamic-ssz types implementHashTreeRoot()but not fastssz'sGetTree()).SSZWithdrawals,Contributions) hash via each element'sHashTreeRoot().+ dynamic-ssz,+ OffchainLabs/go-bitfield,dynssz-gentool.Also here: the §2 Gloas aggregate and attestation on go-eth2-client's Gloas containers
The pinned fork gives
VersionedAttestationandVersionedAggregateAndProoftheir ownGloasmembers, andgloas.Attestationmerkleizes as a progressive container with a progressive-bitlistaggregation_bits: the same bytes as the Electra container, a different hash tree root. The Gloas arms reused the Electra container, so on Gloas slots the aggregate-and-proof signing root would not match a beacon node's (ssvlabs/ssv#3009). The last commit switches every Gloas arm to the Gloas containers:types/consensus_data.go:AggregatorCommitteeConsensusData.Validatedecodesgloas.Attestationfrom Gloas on;GetAggregateAndProofsbuildsgloas.AggregateAndProofinto the wrapper'sGloasfield;GetAggregateAndProofHashRootreturns that field.ssv/aggregator_committee.go: the signed constructor wrapsgloas.SignedAggregateAndProof.ssv/committee_runner.go:ConstructVersionedAttestationWithoutSignatureandVersionedAttestationWithSignatureuse theGloasfield, via the newConstructGloasAttestationWithoutSignature.types/gloas/version.go:DataVersionGloasbecomes an alias of upstream'sspec.DataVersionGloas(same value).types/testingutils: the testing beacon node and the aggregator fixtures build and hash the Gloas containers.The aggregator-committee Gloas vectors change roots and signatures; consensus-data bytes do not. ssvlabs/ssv#2901 pins this branch head and drops its
ForkAtEpochFulu cap on top of it; that pairing is required, since with the Electra-reusing arms the node's Gloas-stamped attestations are rejected by the fork's submit path.Notes & review gates
types/gloas/beacon_block_root_test.godecodes a real devnet Gloas block into both this package'sSignedBeaconBlockand go-eth2-client'sspec/gloastype and asserts equalHashTreeRoot, locking the "byte-identical to the reference" property into CI (round-trip/wire tests alone can't catch a progressive-root drift).make generate-jsons(state-comparison suite) needs the siblingspec-tests/worktree and wasn't run here. When regenerating, confirm only Gloas-fork fixtures change roots and every pre-Gloas fixture stays byte-identical — non-progressive types (BeaconVote,ProposerConsensusData, …) are root-stable under dynssz, but a diff there would be a silent consensus break for deployed forks, so verify rather than assume.Versionstring, so ~620 state-comparison fixtures move from(unknown)- to(gloas)-named files. Thespec-testssync will surface these as deletes + adds (renames), not content diffs — expected, not a regression.IsMaxSize=falseon the two attestation size tests. TheirAggregationBitsis a Bitlist whose byte length can never equal its bit-countssz-max, so the tag check relaxes from==to≤for the whole attestation (the exact size is pinned separately viaExpectedEncodedLength). The flag is exported in the test JSON, so any downstream runner mirroringcheckSSZTagsrelaxes the same way for these two vectors — intended.go.modreplace … => ssvlabs/go-eth2-client@fork-gloasis a reproducible preview pin; can't cut a release until go-eth2-client'sgloaslands upstream. Tracked in ePBS/Gloas: adopt go-eth2-client Gloas support (progressive SSZ types +gloasversion + transport) ssv#3014. Stacked on types: ePBS (Gloas / SIP-94) reference implementation — WIP #633 — merge order matters.tooldirective, needs ≥ 1.24). CI's lint + generate-spec-tests actions are bumped to 1.25.x here; the downstreamssvlabs/ssvmodule is already on go 1.26.BuilderDepositRequestkeeps[32]byte WithdrawalCredentials— same root as go-eth2-client's[]byte ssz-size:32.maxmsgsizecoverage — tracked in ePBS/Gloas: DoS bounds for unbounded progressive-list consensus values (maxmsgsize) ssv#3019.casbin/govaluateenters the graph as dynamic-ssz's expression evaluator even withwithout-dynamic-expressions: true— unavoidable (it's a dynamic-ssz dep).