Skip to content

types: migrate Gloas + consensus-data SSZ to pk910/dynamic-ssz - #643

Draft
iurii-ssv wants to merge 5 commits into
epbs-gloas-typesfrom
epbs-gloas-types-dynssz
Draft

types: migrate Gloas + consensus-data SSZ to pk910/dynamic-ssz#643
iurii-ssv wants to merge 5 commits into
epbs-gloas-typesfrom
epbs-gloas-types-dynssz

Conversation

@iurii-ssv

@iurii-ssv iurii-ssv commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

🚧 Merge blockers: (1) stacked on #633 — merge that first; (2) go.mod pins a preview replace … => ssvlabs/go-eth2-client@fork-gloas — must swap to the upstream gloas release before this merges/releases (tracked in ssvlabs/ssv#3014).

Stacked on #633 (epbs-gloas-types). Migrates the Gloas types — and the consensus-data types that embed go-eth2-client composite types — from ferranbt/fastssz to pk910/dynamic-ssz.

Why

Gloas (EIP-7688 + EIP-7916) makes the affected containers/lists progressive, which fastssz cannot merkleize — so the hand-rolled fastssz encoders computed the wrong hash_tree_root. Correct progressive merkleization has to come from dynamic-ssz, matching go-eth2-client's gloas branch.

What

  • types/gloas — encoders regenerated with dynssz-gen; ssz-index + ssz-type:"progressive-list" tags ported. BeaconBlockBody now uses gloas.Attestation / gloas.AttesterSlashing (not electra).
  • types — consensus-data encoders (BeaconVote, GloasBeaconVote, Contribution, ProposerConsensusData, …) regenerated with dynssz-gen; the rest of the package stays on fastssz. types/generate.go generates the consensus-data types only via dynssz-gen (the overlapping fastssz directive was dropped).
  • go-bitfield prysmaticlabsOffchainLabs (matches go-eth2-client).
  • ssz.HashRoot → a narrow HashRoot interface (dynamic-ssz types implement HashTreeRoot() but not fastssz's GetTree()).
  • Hand-written list helpers (SSZWithdrawals, Contributions) hash via each element's HashTreeRoot().
  • go.mod: go 1.25, + dynamic-ssz, + OffchainLabs/go-bitfield, dynssz-gen tool.

Also here: the §2 Gloas aggregate and attestation on go-eth2-client's Gloas containers

The pinned fork gives VersionedAttestation and VersionedAggregateAndProof their own Gloas members, and gloas.Attestation merkleizes as a progressive container with a progressive-bitlist aggregation_bits: the same bytes as the Electra container, a different hash tree root. The Gloas arms reused the Electra container, so on Gloas slots the aggregate-and-proof signing root would not match a beacon node's (ssvlabs/ssv#3009). The last commit switches every Gloas arm to the Gloas containers:

  • types/consensus_data.go: AggregatorCommitteeConsensusData.Validate decodes gloas.Attestation from Gloas on; GetAggregateAndProofs builds gloas.AggregateAndProof into the wrapper's Gloas field; GetAggregateAndProofHashRoot returns that field.
  • ssv/aggregator_committee.go: the signed constructor wraps gloas.SignedAggregateAndProof.
  • ssv/committee_runner.go: ConstructVersionedAttestationWithoutSignature and VersionedAttestationWithSignature use the Gloas field, via the new ConstructGloasAttestationWithoutSignature.
  • types/gloas/version.go: DataVersionGloas becomes an alias of upstream's spec.DataVersionGloas (same value).
  • types/testingutils: the testing beacon node and the aggregator fixtures build and hash the Gloas containers.

The aggregator-committee Gloas vectors change roots and signatures; consensus-data bytes do not. ssvlabs/ssv#2901 pins this branch head and drops its ForkAtEpoch Fulu cap on top of it; that pairing is required, since with the Electra-reusing arms the node's Gloas-stamped attestations are rejected by the fork's submit path.

Notes & review gates

  • HTR invariant is testedtypes/gloas/beacon_block_root_test.go decodes a real devnet Gloas block into both this package's SignedBeaconBlock and go-eth2-client's spec/gloas type and asserts equal HashTreeRoot, locking the "byte-identical to the reference" property into CI (round-trip/wire tests alone can't catch a progressive-root drift).
  • Fixture regen is a review gate. make generate-jsons (state-comparison suite) needs the sibling spec-tests/ worktree and wasn't run here. When regenerating, confirm only Gloas-fork fixtures change roots and every pre-Gloas fixture stays byte-identical — non-progressive types (BeaconVote, ProposerConsensusData, …) are root-stable under dynssz, but a diff there would be a silent consensus break for deployed forks, so verify rather than assume.
  • Fixture renames in the spec-tests sync. The Gloas fork now supplies the beacon Version string, so ~620 state-comparison fixtures move from (unknown)- to (gloas)-named files. The spec-tests sync will surface these as deletes + adds (renames), not content diffs — expected, not a regression.
  • IsMaxSize=false on the two attestation size tests. Their AggregationBits is a Bitlist whose byte length can never equal its bit-count ssz-max, so the tag check relaxes from == to for the whole attestation (the exact size is pinned separately via ExpectedEncodedLength). The flag is exported in the test JSON, so any downstream runner mirroring checkSSZTags relaxes the same way for these two vectors — intended.
  • Temporary fork pin = merge blocker. go.mod replace … => ssvlabs/go-eth2-client@fork-gloas is a reproducible preview pin; can't cut a release until go-eth2-client's gloas lands upstream. Tracked in ePBS/Gloas: adopt go-eth2-client Gloas support (progressive SSZ types + gloas version + transport) ssv#3014. Stacked on types: ePBS (Gloas / SIP-94) reference implementation — WIP #633 — merge order matters.
  • Go floor 1.22 → 1.25 (+ tool directive, needs ≥ 1.24). CI's lint + generate-spec-tests actions are bumped to 1.25.x here; the downstream ssvlabs/ssv module is already on go 1.26.
  • BuilderDepositRequest keeps [32]byte WithdrawalCredentials — same root as go-eth2-client's []byte ssz-size:32.
  • Follow-up (not this PR): progressive lists are now unbounded (EIP-7916), so the Gloas consensus values need DoS size bounds + maxmsgsize coverage — tracked in ePBS/Gloas: DoS bounds for unbounded progressive-list consensus values (maxmsgsize) ssv#3019.
  • Supply-chain note: casbin/govaluate enters the graph as dynamic-ssz's expression evaluator even with without-dynamic-expressions: true — unavoidable (it's a dynamic-ssz dep).

@greptile-apps

greptile-apps Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR migrates Gloas and consensus-data SSZ encoding and hashing from fastssz to dynamic-ssz, updates the associated beacon types and signing interface, and changes supporting dependencies.

  • Adds generated dynamic-ssz encoders for consensus-data and Gloas containers, including progressive structures.
  • Narrows the runner-facing hash interface to HashTreeRoot() and updates signing and root-validation paths.
  • Switches bitfield implementations, pins a Gloas-capable go-eth2-client fork, and adds dynssz-gen tooling.
  • Leaves the old consensus-data fastssz generation step active alongside the new generator.

Confidence Score: 4/5

The runtime migration appears coherent, but the PR should not merge until consensus-data generation uses only the new dynamic-ssz path.

Running the repository’s generation command still invokes fastssz for the migrated consensus-data structs before dynssz-gen generates the same SSZ method set, making checked-in encoder regeneration fail or produce conflicting declarations.

Files Needing Attention: types/generate.go

Important Files Changed

Filename Overview
types/generate.go Adds dynssz-gen for consensus-data but retains the overlapping legacy fastssz generator, breaking regeneration.
types/consensus_data.go Updates consensus-data types and bridges composite-list hashing through each dynamic type’s canonical root.
types/ssz.go Introduces the narrow HashRoot interface and adapts withdrawals-list hashing for dynamic-ssz elements.
types/gloas/beaconblockbody_ssz.go Provides generated dynamic-ssz encoding and progressive merkleization for the Gloas block body.
ssv/proposer.go Routes Gloas proposer blocks through the new decoder and HashRoot signing path.
go.mod Raises the Go version, adds dynamic-ssz tooling and dependencies, changes bitfield implementation, and pins the preview go-eth2-client fork.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  S[Consensus and Gloas source types] --> D[dynssz-gen]
  D --> E[Dynamic SSZ encoders and roots]
  E --> C[Consensus values]
  C --> R[Duty runners]
  R --> H[HashTreeRoot]
  H --> P[Partial signatures and beacon submission]
  S -. legacy overlapping path .-> F[fastssz sszgen]
  F -. duplicate consensus-data methods .-> E
Loading

Comments Outside Diff (1)

  1. types/generate.go, line 18-25 (link)

    P1 Overlapping SSZ generators break regeneration

    When go generate ./types runs, it first regenerates the migrated consensus-data methods with fastssz and then generates the same method set with dynssz-gen, causing generation failures or duplicate declarations in the package.

    Knowledge Base Used: Restore dependency and code-generation compatibility

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Reviews (1): Last reviewed commit: "types: migrate Gloas + consensus-data SS..." | Re-trigger Greptile

@iurii-ssv
iurii-ssv force-pushed the epbs-gloas-types-dynssz branch 2 times, most recently from 658d6b1 to e6e01e5 Compare September 1, 2026 13:56
@iurii-ssv
iurii-ssv force-pushed the epbs-gloas-types-dynssz branch from e6e01e5 to 05dea41 Compare September 1, 2026 14:30
iurii-ssv added a commit that referenced this pull request Sep 2, 2026
maxmsgsize: build the phase0/electra max-attestation fixtures with valid
bitlists (bitfield.NewBitlist) so they encode under dynamic-ssz. A bitlist's
byte length can't equal its bit-count ssz-max, so these structure-size tests
now check ≤ and pin the exact size (485 / 16621) via ExpectedEncodedLength;
MaxSize{Phase0,Electra}Attestation are documented as intentional
over-approximations feeding the AggregatedAttestations list limit.

generate: resolve the go-eth2-client include via `go list -m` (wrapped in sh -c)
instead of the hard-coded $GOPATH/...@v0.27.0 path, which no longer exists under
the fork replace. Drop the no-op gloas `rm -f *_ssz.go` (go generate never
expanded the glob) and document that dynssz-gen overwrites the encoders in
place: it must type-check the package to load it, and the hand-written
Encode/Decode wrappers reference the generated MarshalSSZ, so the files can't be
deleted first — a stale encoder for a removed type fails the build instead.

gloas: note that ssv-spec keeps its own Gloas containers (spec-owned, pinned to
the fork by beacon_block_root_test.go) rather than aliasing the fork; align
ExecutionPayloadBid.PrevRandao with the fork (phase0.Root, same SSZ); fix a
devnet-6/devnet-8 comment.
@iurii-ssv
iurii-ssv force-pushed the epbs-gloas-types-dynssz branch from 4e04768 to c719c7d Compare September 2, 2026 08:17
@iurii-ssv

Copy link
Copy Markdown
Contributor Author

Re: the "Overlapping SSZ generators break regeneration" P1 — this was reviewed against an earlier commit and is already resolved at HEAD. types/generate.go no longer runs fastssz sszgen for the consensus-data types; they are generated only by dynssz-gen -config generate_cd.yaml. make generate-ssz now runs clean and git status stays clean after regeneration (no duplicate declarations).

ssvlabs-spec-test-sync Bot added a commit to ssvlabs/spec-tests that referenced this pull request Sep 2, 2026
iurii-ssv added a commit that referenced this pull request Sep 2, 2026
maxmsgsize: build the phase0/electra max-attestation fixtures with valid
bitlists (bitfield.NewBitlist) so they encode under dynamic-ssz. A bitlist's
byte length can't equal its bit-count ssz-max, so these structure-size tests
now check ≤ and pin the exact size (485 / 16621) via ExpectedEncodedLength.
MaxSizeElectraAttestation (which bounds the AggregatedAttestations list element)
is documented as an intentional over-approximation; the now-unused
MaxSizePhase0Attestation is dropped.

generate: resolve the go-eth2-client include via `go list -m` (wrapped in sh -c)
instead of the hard-coded $GOPATH/...@v0.27.0 path, which no longer exists under
the fork replace. Drop the no-op gloas `rm -f *_ssz.go` (go generate never
expanded the glob) and document that dynssz-gen overwrites the encoders in
place: it must type-check the package to load it, and the hand-written
Encode/Decode wrappers reference the generated MarshalSSZ, so the files can't be
deleted first — a stale encoder for a removed type fails the build instead.

gloas: note that ssv-spec keeps its own Gloas containers (spec-owned, pinned to
the fork by beacon_block_root_test.go) rather than aliasing the fork; align
ExecutionPayloadBid.PrevRandao with the fork (phase0.Root, same SSZ); fix a
devnet-6/devnet-8 comment.
@iurii-ssv
iurii-ssv force-pushed the epbs-gloas-types-dynssz branch from c719c7d to 95a0124 Compare September 2, 2026 08:39
ssvlabs-spec-test-sync Bot added a commit to ssvlabs/spec-tests that referenced this pull request Sep 2, 2026
ssvlabs-spec-test-sync Bot added a commit to ssvlabs/spec-tests that referenced this pull request Sep 2, 2026
iurii-ssv added a commit to ssvlabs/ssv that referenced this pull request Sep 2, 2026
… roots

Aggregator duties failed on Gloas slots with "unknown data version: 8" (#3009): the fork already
decodes the gloas Eth-Consensus-Version into VersionedAttestation.Gloas and ships a distinct
gloas.AggregateAndProof (its Attestation merkleizes progressively, so its root differs from the
byte-identical Electra container's), but the node's version switches had no Gloas arm. Add it to the
goclient aggregate conversions, the consensus-data decode, and a signed-aggregate constructor now
shared by both aggregator runners; the remote signer rejects Gloas aggregates explicitly (#3000).

Drop ForkAtEpoch's Fulu cap in the node and ssvsigner: a scheduled Gloas fork resolves from its epoch
on and absent forks are skipped, so attestations ride gloas.Attestation and the aggregator-committee
consensus data stamps DataVersionGloas (SIP #94 §2, #2998). This needs ssv-spec's Gloas arms to build
the Gloas containers, so both go.mods repin to the ssv-spec dynssz branch head that carries them
(ssvlabs/ssv-spec#643). The remote signer's fork_info substitution is no longer needed.

Pin the Gloas roots to the chain (#3008): a devnet-8 finalized block whose HashTreeRoot must equal the
chain's header root and whose proposer signature must verify, plus its execution-payload envelope,
which must blind to the full envelope's root. EIP-7688/7916 change only merkleization, so round-trip
tests alone cannot catch a wrong root.
@iurii-ssv
iurii-ssv marked this pull request as draft September 4, 2026 11:39
iurii-ssv added a commit that referenced this pull request Sep 4, 2026
maxmsgsize: build the phase0/electra max-attestation fixtures with valid
bitlists (bitfield.NewBitlist) so they encode under dynamic-ssz. A bitlist's
byte length can't equal its bit-count ssz-max, so these structure-size tests
now check ≤ and pin the exact size (485 / 16621) via ExpectedEncodedLength.
MaxSizeElectraAttestation (which bounds the AggregatedAttestations list element)
is documented as an intentional over-approximation; the now-unused
MaxSizePhase0Attestation is dropped.

generate: resolve the go-eth2-client include via `go list -m` (wrapped in sh -c)
instead of the hard-coded $GOPATH/...@v0.27.0 path, which no longer exists under
the fork replace. Drop the no-op gloas `rm -f *_ssz.go` (go generate never
expanded the glob) and document that dynssz-gen overwrites the encoders in
place: it must type-check the package to load it, and the hand-written
Encode/Decode wrappers reference the generated MarshalSSZ, so the files can't be
deleted first — a stale encoder for a removed type fails the build instead.

gloas: note that ssv-spec keeps its own Gloas containers (spec-owned, pinned to
the fork by beacon_block_root_test.go) rather than aliasing the fork; align
ExecutionPayloadBid.PrevRandao with the fork (phase0.Root, same SSZ); fix a
devnet-6/devnet-8 comment.
@iurii-ssv
iurii-ssv force-pushed the epbs-gloas-types-dynssz branch from b894983 to f0772bc Compare September 4, 2026 15:49
ssvlabs-spec-test-sync Bot added a commit to ssvlabs/spec-tests that referenced this pull request Sep 4, 2026
Gloas (EIP-7688/7916) makes the affected containers/lists progressive, which
fastssz cannot merkleize, so the hand-rolled fastssz encoders computed wrong
hash_tree_roots. Move the Gloas types and the consensus-data types that embed
go-eth2-client composite types onto pk910/dynamic-ssz (matching go-eth2-client's
gloas branch) and reconcile the fastssz -> dynamic-ssz interface change:

- types/gloas: regenerate encoders with dynssz-gen; port ssz-index +
  ssz-type:"progressive-list" tags. BeaconBlockBody now uses gloas.Attestation /
  gloas.AttesterSlashing (not electra).
- types: regenerate the consensus-data encoders (BeaconVote, GloasBeaconVote,
  Contribution, ProposerConsensusData, ...) with dynssz-gen; the rest of the
  package stays on fastssz. types/generate.go generates the consensus-data
  types only via dynssz-gen (dropped the overlapping fastssz directive).
- go-bitfield: prysmaticlabs -> OffchainLabs (matches go-eth2-client).
- ssz.HashRoot -> a narrow HashRoot interface (dynamic-ssz types implement
  HashTreeRoot() but not fastssz's GetTree()).
- Hand-written list helpers (SSZWithdrawals, Contributions) hash via each
  element's HashTreeRoot() instead of fastssz's HashTreeRootWith.
- Add committed HTR-vs-go-eth2-client regression tests (types/gloas): a real
  devnet block (progressive container) and a non-empty progressive list, each
  decoded into both types and asserted equal — pinning progressive merkleization
  in CI, independent of the external spec-tests repo.
- Refresh beacon_types_encoding.go (fastssz Hash reflects the resolved fork) so
  `go generate ./types/` is idempotent; document the generated (*_ssz.go) vs
  hand-written (snake_case.go) split in types/gloas/generate.go.
- go.mod: go 1.25, + dynamic-ssz, + OffchainLabs/go-bitfield, dynssz-gen tool,
  temporary replace to ssvlabs/go-eth2-client@fork-gloas until gloas releases.
- CI: bump lint + generate-spec-tests actions to Go 1.25.x for the go.mod floor
  and the tool directive.

Stacked on epbs-gloas-types. BuilderDepositRequest keeps [32]byte
WithdrawalCredentials (same root as go-eth2-client's []byte ssz-size:32).
BeaconVoteDataNil builds a BeaconVote with nil Source/Target, then passed
*consensusData.Source / *consensusData.Target to NewSpecTest, which takes
phase0.Checkpoint by value. That deref only worked by accident: fastssz's
MarshalSSZ allocated the nil checkpoints on the receiver, so by the time the
derefs ran Source/Target were non-nil zero checkpoints. This package's move to
dynamic-ssz — whose MarshalSSZ does not mutate the receiver — leaves them nil,
so the derefs panic and take down the whole vector generator (no ssv spec-test
vectors could be produced).

Pass zero checkpoints explicitly: a nil Source/Target SSZ-encodes to a zero
checkpoint, so that is the value the decoded vote carries, and the test still
asserts the source-not-less-than-target rejection.
maxmsgsize: build the phase0/electra max-attestation fixtures with valid
bitlists (bitfield.NewBitlist) so they encode under dynamic-ssz. A bitlist's
byte length can't equal its bit-count ssz-max, so these structure-size tests
now check ≤ and pin the exact size (485 / 16621) via ExpectedEncodedLength.
MaxSizeElectraAttestation (which bounds the AggregatedAttestations list element)
is documented as an intentional over-approximation; the now-unused
MaxSizePhase0Attestation is dropped.

generate: resolve the go-eth2-client include via `go list -m` (wrapped in sh -c)
instead of the hard-coded $GOPATH/...@v0.27.0 path, which no longer exists under
the fork replace. Drop the no-op gloas `rm -f *_ssz.go` (go generate never
expanded the glob) and document that dynssz-gen overwrites the encoders in
place: it must type-check the package to load it, and the hand-written
Encode/Decode wrappers reference the generated MarshalSSZ, so the files can't be
deleted first — a stale encoder for a removed type fails the build instead.

gloas: note that ssv-spec keeps its own Gloas containers (spec-owned, pinned to
the fork by beacon_block_root_test.go) rather than aliasing the fork; align
ExecutionPayloadBid.PrevRandao with the fork (phase0.Root, same SSZ); fix a
devnet-6/devnet-8 comment.
…ient's Gloas containers

go-eth2-client's Gloas types give VersionedAttestation and VersionedAggregateAndProof their own Gloas
members, and gloas.Attestation merkleizes as a progressive container with a progressive-bitlist
aggregation_bits (EIP-7688 / EIP-7916): the same bytes as the Electra container, a different hash tree
root. The Gloas arms here reused the Electra container, so on Gloas slots the aggregate-and-proof
signing root would not match a beacon node's (ssvlabs/ssv#3009).

Switch every Gloas arm to the Gloas containers: AggregatorCommitteeConsensusData.Validate and
GetAggregateAndProofs, GetAggregateAndProofHashRoot, the aggregator-committee signed constructor, the
committee runner's attestation helpers (new ConstructGloasAttestationWithoutSignature), and the
testing beacon node and aggregator fixtures. DataVersionGloas becomes an alias of upstream's
spec.DataVersionGloas (same value). The aggregator-committee Gloas vectors change roots and
signatures; consensus-data bytes do not.
Fold the §5 BuilderRequestAuth / SignedBuilderRequestAuth types into the
pk910/dynamic-ssz generation, matching the rest of the Gloas package on this
branch: drop the fastssz sszgen directive and its _encoding.go output, add the
two types to generate.yaml, and regenerate the flatcase *_ssz.go encoders. The
slot runner's expectedRequestAuthRootsAndDomain returns the narrow
types.HashRoot (dynssz types implement HashTreeRoot but not fastssz's GetTree).
@iurii-ssv
iurii-ssv force-pushed the epbs-gloas-types-dynssz branch from f0772bc to f461f5b Compare September 4, 2026 18:04
ssvlabs-spec-test-sync Bot added a commit to ssvlabs/spec-tests that referenced this pull request Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant