-
Notifications
You must be signed in to change notification settings - Fork 2.2k
refactor(trie): consolidate StorageTries parallel maps into one #20714
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
crates/trie/sparse/src/state.rs
Outdated
| /// Takes the storage trie for the provided address. | ||
| pub fn take_storage_trie(&mut self, address: &B256) -> Option<SparseTrie<S>> { | ||
| self.storage.tries.remove(address) | ||
| self.storage.entries.remove(address).map(|e| e.trie) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Difference:
- Old: Only removes trie, revealed_paths for this account persists
- New: Removes both trie and revealed_paths
This is a behavior change. If something called take_storage_trie and expected revealed_paths to still be there, it would break.
not 100% sure about this, defer to @mediocregopher or @shekhirin
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch, this will indeed break other code. This method is used in two places, and they both do the same thing:
- Call
take_storage_trieon an account - Modify the storage trie
- Put the account's storage trie back using
insert_storage_trie
Given this change the first step will discard the revealed_paths, such that on the third step the trie will appear to have no revealed paths.
I think the best thing to do would be to change these methods to take_storage_trie_entry and insert_storage_trie_entry. It will result in a larger PR but I think it's the cleanest solution.
| .or_insert_with(|| self.cleared_revealed_paths.pop().unwrap_or_default()); | ||
|
|
||
| (trie, revealed_paths) | ||
| let entry = self.get_entry_mut(account); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Difference in creation logic:
- Old: For a new account, tries to pop from cleared_tries, else clones default_trie. Separately, tries to pop from
cleared_revealed_paths, else creates empty HashSet. - New: Tries to pop from cleared_entries (which has BOTH trie and paths), else creates new entry with cloned
default_trieand empty HashSet.
Potential issue: In the old code, cleared_tries and cleared_revealed_paths were independent pools. we could have 10 cleared tries and 5 cleared paths. In the new code, they're always paired in cleared_entries.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Imo this should be fine, I don't believe there was a situation where the length of revealed paths and tries was different in the old code.
yongkangc
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is promising, however regarding the trie logic would defer to @mediocregopher or @shekhirin to take a closer look at it
mattsse
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@shekhirin is this similar to your arena wip?
| .or_insert_with(|| self.cleared_revealed_paths.pop().unwrap_or_default()); | ||
|
|
||
| (trie, revealed_paths) | ||
| let entry = self.get_entry_mut(account); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Imo this should be fine, I don't believe there was a situation where the length of revealed paths and tries was different in the old code.
crates/trie/sparse/src/state.rs
Outdated
| /// Takes the storage trie for the provided address. | ||
| pub fn take_storage_trie(&mut self, address: &B256) -> Option<SparseTrie<S>> { | ||
| self.storage.tries.remove(address) | ||
| self.storage.entries.remove(address).map(|e| e.trie) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch, this will indeed break other code. This method is used in two places, and they both do the same thing:
- Call
take_storage_trieon an account - Modify the storage trie
- Put the account's storage trie back using
insert_storage_trie
Given this change the first step will discard the revealed_paths, such that on the third step the trie will appear to have no revealed paths.
I think the best thing to do would be to change these methods to take_storage_trie_entry and insert_storage_trie_entry. It will result in a larger PR but I think it's the cleanest solution.
Co-authored-by: Brian Picciano <[email protected]>
StorageTrieEntryfor better cache localitythis makes storage related lookups in the sparse trie 2x faster and we also save 48 bytes per
StorageTrieinstancebenchmark results: