Skip to content

Run execution witness tests against official releases (& debugging tips) #18

@jsign

Description

@jsign

The current approach (#8) is to pass some execution witness tests that were experimental from some months ago. It is highly recommended that the codebase switch to the new official releases.

The execution-specs@projects/zkevm is the branch which as:

  • The specs on how execution witness must be generated in a block.
  • Proper testing framework changes to fill all existing EEST tests to generate the executionWitness field in the fixture.

Apart from filling all (>16k) tests with the new executionWitness field, the branch also adds specific (currently ~140) tests which:

  • Generates very particular scenarios for execution witness generation:
    • bytecodes should properly work in all relevant cases e.g. system contracts, EXTCODECOPY, reverted call frames bytecode access, etc
    • headers should properly work for system contract EIP-2935 needs, and multiple BLOCKHASH situations that could raise.
    • state should properly be generated in proof of absence, presence, and also post-state root branch compression scenarios.

The recommendation is to use the releases mentioned to run the execution-witness CI tests for Reth. The `tests/amsterdam/eip8025_optional_proofs are the most focused cases to start with (i.e., newly focused, created cases for execution witness), and let the rest (~16k) cases simply be more EEST cases, assisting cases that could also uncover other situations. As in, the former tests are clearly designed to cover a specific angle of execution witness generation, thus a mismatch in the test is easy to understand the reason.

Extra: debugging tips

In case you need to debug manually (or provide tight feedback loop to LLMs):

  1. Clone the execution-specs repo
  2. Switch to branch projects/zkevm-bal-devnet-3 -- always latest stable branch used for the latest official release.
  3. Run uv run fill --clean -m "blockchain_test" --fork Amsterdam -s ./tests/amsterdam/eip8025_optional_proofs -k <some test name filter>

You can change the ./tests folder path to any other folder you want, and use -k to filter specific tests from there. (Not using -k will fill all tests in tha file).

This is useful to potentially add prints (e.g. here for tree related stuff, or look for the state tracker, or similar. Always in the amsterdam spec folder) to the spec at any point, and let the -s be shown when the test is filled. This can help explain why particular items are added to execution witness fields (e.g., state, headers, bytecodes). This is quite useful for debugging why an EL might not be matching what the spec is doing, and for easily comparing both for failing cases, etc. Take this approach as something useful I’ve done to accelerate integration — provides a quick feedback loop to debug faster.

If, for whatever reason, while debugging some failing fixture, you think the specs have a bug, please don’t hesitate to get in contact with me. While we have tested this against some ELs, we are still in early days, and there might be spec bugs.

cc @figtracer

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions