perf: execution and tracegen rewrite #1567
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jonathanpwang
force-pushed
the
feat/new-execution
branch
from
ea0bdd9 to
ea3775c
Compare
Note: this PR is not targeting `main`. I've used `TODO` and `TEMP` to mark places in code that will need to be cleaned up before merging to `main`. Beginning the refactor of online memory to allow different host types in different address spaces. Going to touch a lot of APIs. Focusing on stabilizing APIs - currently this PR will not improve performance. Tests will not all pass because I have intentionally disabled some logging required for trace generation. Only execution tests will pass (or run the execute benchmark). In future PR(s): - [ ] make `Memory` trait for execution read/write API - [ ] better handling of type conversions for memory image - [ ] replace the underlying memory implementation with other implementations like mmap Towards INT-3743 Even with wasteful conversions, execution is faster: Before: https://github.com/openvm-org/openvm/actions/runs/14318675080 After: https://github.com/openvm-org/openvm/actions/runs/14371335248?pr=1559
- make `VmSegmentExecutor` generic on `<Mem, Ctx, Ctrl>` where: - `Mem`: struct that implements `GuestMemory` - `Ctx`: struct that stores host context during execution - `Ctrl`: struct that implements pre/post segment execution hooks, termination condition and instruction execution logic - add `TracegenVmSegmentExecutor` that implements the current execution flow - move segmentation strategies to new file
- deleting `Vm{Adapter,Core}Chip` traits
- no more records, directly use trace buffer
- jal_lui chip is a demonstration of the new changes with working unit
tests
- changed unit tester
- [x] need to add some dummy volatile memory to the tester to balance
based on touched addresses
…1590) - introduce a new generic `InsExecutorE1` trait - add `InsExecutor::execute_e1` for rv32im instructions
…1589) Co-authored-by: Alexander Golovanov <[email protected]>
- fix some loadstore tests - remove records - wrap unsafe memory read/writes into safe wrappers --------- Co-authored-by: Jonathan Wang <[email protected]>
closes INT-3839 --------- Co-authored-by: Ayush Shukla <[email protected]>
- make `Rv32HintStoreChip` use the `NewVmChipWrapper` - rename `SingleTraceStep` to `TraceStep` and update it to work for chips whose execution creates multiple trace rows - comment out criterion execute benchmarks for now
one line fix. now that we're only initializing `TracingMemory` with `new`, we should remove this line from `with_image`
jonathanpwang
force-pushed
the
feat/new-execution
branch
from
6f764a4 to
4a2ac13
Compare
remove `memory/offline.rs` as we aren't using it anymore. Delete `VmAdapterChip` trait and `VmChipWrapper` since we also aren't using them anymore.
Made the rv32im tests pass and made all the testing files to have the same testing interface. Deleted the `test_adapter`. Kept all the test cases unchanged. The only commented test case remaining is the `store` test to the address space 4, which is failing because currently memory accesses with block size 4 are not supported with the address space 4. All the test files have 3 types of tests: Positive, Negative, and Sanity tests. All the test files have 2 helper functions: `create_test_chip`, `set_and_execute`. An important thing to notice about negative tests when expecting an interaction fail (aka ChallangePhase error) is that ther might be an imbalance created for the wrong reasons. For example, there might be an imbalance on the range checker bus created by the interactions: [send 1] (sent from the chip_air) [receive 2] (the execution did `add_count(2)` at some point) This is not a "valid" fail since 1 is still in the range of the range checker. Because of this a manual check is needed for all the negative checks. To see all the imbalances occurred during a test remove the 'disable_debug_builder();' line from the `run_negative_test` function and run the test. I am 95% sure that I wen through all the negative tests and checked that the imbalances occurred are correct. The `test_adapter` tried to address this issue by getting rid of interaction imbalances on the memory bus. But even with the `test _adapter` a manual check was necessary. To solve this I suggest that we somehow keep all the interactions that occur during the test and automatically check that actually an invalid interaction has happened on a specified bus. Resolves INT-3975 --------- Co-authored-by: Ayush Shukla <[email protected]>
Fixed an error in divrem negative tests. The trace pranking was done incorrectly. 2 instructions were being called (so the trace had height 2) each time but only one of the rows was being modified. Changed it so only one instruction is called each time Also, made the setup_tracing the default
Implemented e1 and e3 for HeapBranch, Heap, and VecHeap adapters. Updated the Bigint circuit correspondingly. Had to make some changes in the interfaces of rv32im Steps. In particular - Changed Reads type `([u8; N], [u8; N])` into `Into<[[u8;N];2]>` and Writes type `[u8; N]` into `From<[[u8;N];1]>`. This change corresponds to what we used to do with the previous integration API in order to make the interfaces to match. - Got rid of TraceAdapterContext in a lot of places. This is because the same Step can be using different AdapterSteps that require different TraceContexts. Or even the AdapterStep might require a `TraceContext` that the Step doesn't have. The easy solution was to implement AdapterSteps in a similar way as in the previous integration API. That is, added the necessary fields to the AdapterStep structs. I am thinking maybe deleting the `TraceContext` from the interface makes sense. I am not sure if there is a better way to do this Important Note: the tests don't run right now because a lot of the read/write operations are done in address space 2 with block size 32 but currently only block size 4 is supported by the memory. Resolves INT-3980
Closes INT-4013
Resolves INT-3801. - Added memory access adapters. To improve: * Allocate the trace buffer once before filling it as opposed to pushing to `Vec` how it's done now, * Maybe not call `get_f` too often (although I don't know how to avoid it normally). - Added volatile and persistent boundary chips tracegen, - Added merkle chip tracegen as described [here](https://docs.google.com/document/d/12cH7ZYRFWHgflpPzOILb7bg5XExdyWOL4vwrQ9HFGkQ/edit?tab=t.0#heading=h.hrg0oexxgu9). To improve: * Parallelize at least something, * Maybe support passing this struct between segments. - `VmChipTestBuilder` now has `::default_persistent`, so all tests in `extensions/rv32im/circuit` pass both with volatile and persistent memory interface.
`cargo` complains that `uuid` has a conflict checksum.
I used to handle creating new blocks in a wrong way when `align > initial_block_size`, now I hopefully do it right. Also added persistent base alu tests, although nothing changed for the persistent case, and added a dummy access in all of them that used to fail.
CodSpeed Instrumentation Performance ReportMerging #1567 will not alter performanceComparing Summary
|
This resolves INT-4012 by not using memory controller's memory in E1 execution.
implemented e1 and e3 for `VecHeapTwoReads` and `eq_mod` rv32 adapters. Implemented e1 and e3 for mod-builder. Updated the `algebra` and `ecc` extensions accordingly. Deleted all the pairing chips All the tests successfully run. Also, added back the address space 4 loadstore tests. Resolves INT-3914
- add codspeed walltime measurement job - tweak execution benchmarks to be heavier and more representative
This comment was marked as outdated.
This comment was marked as outdated.
…sts) (#1659) This resolves INT-3913. As a _side effect_, this removes `GuestMemory` trait -- it is a struct now with underlying `AddressMap<PAGE_SIZE>` (I didn't make `type GuestMemory = ...` because the vaguely called `read` and `write` methods would be too vaguely called for `AddressMap`). `VmStateMut` is generic over `MEM` though. I didn't fully implement `TraceStep` and `StepExecutorE1` for the phantom chip because the chip is relatively easy and I'm not sure it would be better expressible in terms of `NewVmChipWrapper`. `PhantomSubExecutor` also changed a little (now accepts `u32` instead of `F`, for example, and also `GuestMemory` instead of what it needed before).
breaking change: `SystemConfig::default()` now has `continuation_enabled: true` (previously it was `false`). closes INT-4526 INT-4525
I was going to rename `max_segment_length`, but it was in too many places of the CI so I didn't bother.
This comment was marked as outdated.
This comment was marked as outdated.
jonathanpwang
changed the title
There was a problem hiding this comment.
Pull Request Overview
This PR introduces a comprehensive rewrite of the VM execution and trace generation architecture, specifically focusing on performance improvements through interpreter-based execution and optimized tracegen. The changes introduce multiple execution modes (E1/E2/E3) and redesign the chip complex system to support different backends and arenas.
Key changes:
- Introduces a new interpreted execution system with pre-computed function pointers for performance
- Redesigns the VM extension system with separate traits for execution, circuit, and prover extensions
- Restructures memory configuration and adds support for multiple address spaces with different cell types
Reviewed Changes
Copilot reviewed 157 out of 518 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| crates/vm/src/arch/interpreter.rs | New interpreter implementation with pre-computed function pointers for E1/E2 execution |
| crates/vm/src/arch/integration_api.rs | Redesigned integration API with new trace filler patterns and chip wrapper abstractions |
| crates/vm/src/arch/hasher/mod.rs | Updated hasher trait to be thread-safe with Send + Sync bounds |
| crates/vm/src/arch/extensions.rs | Major restructuring of VM extension system with separate execution/circuit/prover traits |
| crates/vm/src/arch/execution_mode/ | New execution mode system with pure, metered, and metered cost variants |
| crates/vm/src/arch/execution.rs | Updated execution traits and error handling for new interpreter system |
| crates/vm/src/arch/config.rs | Enhanced configuration system with address space management and memory cell types |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Closes INT-4644, INT-4645, INT-4650, INT-4648 (links are checked automatically by vocs)
Testing: - Verified on clean Ubuntu install - Verified on clean MacOS install with `xcode-select --install` already run Closes INT-4585
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Reduces memory fragmentation by returning pages to OS with some delay. Credited to @nyunyunyunyu
This comment was marked as outdated.
This comment was marked as outdated.
- emit a warning if `max_constraint_degree` don't matching between `VmConfig` and `StarkFriConfig`
Commit: f73da4d |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To be squash merged
What's Changed
See changelog for details.
The STARK backend
MultiStarkVerifyingKeys for all existing VM configs remains unchanged between v1.3.0 and this PR, with the exception of theRootVerifierProvingKeynoted below. However theAppVerifyingKeyhas a difference in binary serialization due to the removal of anas_offsetfield fromMemoryDimensions. This workflow shows that after converting to account for this change:AppVerifyingKeys remain unchangedHalo2Config::default()now usesverifier_k = 23instead ofverifier_k = 24for a smaller default halo2 circuit. The initial Fiat-Shamir transcript state also changed due to the Root Verifier vkey change above.