[DO NOT MERGE] Test pre-fork functional tests#111
Draft
j-berman wants to merge 42 commits intoseraphis-migration:fcmp++-stagefrom
Draft
[DO NOT MERGE] Test pre-fork functional tests#111j-berman wants to merge 42 commits intoseraphis-migration:fcmp++-stagefrom
j-berman wants to merge 42 commits intoseraphis-migration:fcmp++-stagefrom
Conversation
This replaces `ver_rct_non_semantics_simple_cached()` with an API that offloads the responsibility of tracking input verification successes to the caller. The main caller of this function in the codebase, `cryptonote::Blockchain()` instead keeps track of the verification results for transaction in the mempool by storing a "verification ID" in the mempool metadata table (with `txpool_tx_meta_t`). This has several benefits, including: * When the mempool is large (>8192 txs), we no longer experience cache misses and unnecessarily re-verify ring signatures. This greatly improves block propagation time for FCMP++ blocks under load * For the same reason, reorg handling can be sped up by storing verification IDs of transactions popped from the chain * Speeds up re-validating every mempool transaction on fork change (monerod revalidates the whole tx-pool on HFs monero-project#10142) * Caches results for every single type of Monero transaction, not just latest RCT type * Cache persists over a node restart * Uses 512KiB less RAM (8192*2*32B) * No additional storage or DB migration required since `txpool_tx_meta_t` already had padding allocated * Moves more verification logic out of `cryptonote::Blockchain` Furthermore, this opens the door to future multi-threaded block verification speed-ups. Right now, transactions' input proof verification is limited to one transaction at a time. However, one can imagine a scenario with verification IDs where input proofs are optimistically multi-threaded in advance of block processing. Then, even though ring member fetching and verification is single-threaded inside of `cryptonote::Blockchain::check_tx_inputs()`, the single thread can skip the CPU-intensive cryptographic code if the verification ID allows it. Also changes the default log category in `tx_verification_utils.cpp` from "blockchain" to "verify".
Co-authored-by: j-berman <justinberman@protonmail.com>
Co-authored-by: jeffro256 <jeffro256@tutanota.com> Co-authored-by: Luke Parker <lukeparker5132@gmail.com> Co-authored-by: SyntheticBird45 <someoneelse.is_on.github.rio7x@simplelogin.com>
jeffro256
force-pushed
the
fcmp++-stage
branch
from
29f12ff to
7bfffa9
Compare
Otherwise we can end up double counting txs towards the weight, which can over-state the pool weight. E.g. relay tx to node in stem phase, add its weight to pool weight, then receive tx from another node, then bump the pool weight again. That double counts the tx towards the pool weight. If the weight exceeds the max, the node will "prune" txs from the pool. Thus, over-counting is probably a cause of, but perhaps not the only cause of: seraphis-migration#148
Curve Trees: handle get_max_concurrency() == 0
tx pool: only increment m_txpool_weight for newly added pool txs
Fixes pruning the database under FCMP++ and prevents future corruption by checking the version value inside the properties table.
Log total pool weight
…prune blockchain_prune: add FCMP tables and check DB version
…ver-ids Fix FCMP++ batch verification collecting ver ID's
fcmp++ rust: ASAN support
Also add documention.
…arrot_devs carrot_impl: refactor scanning_tools to use Carrot devices
…rint cryptonote_basic: remove BP+ clawback debug print in weight func
…d_designator carrot_impl: specify all fields in aggregate init (1)
…ix_usage_fcmp++-stage serialization: revert va_args_commaprefix usage
…bytes fcmp++: use weight approx equal to byte size
carrot_impl: device overhaul
fcmp++: fix expand_transaction_2(<already expanded tx>)
…rialization_stage Fix fcmp++ block serialiazationin ZMQ
fcmp_pp: more readable container ops in get_dummy_paths()
j-berman
force-pushed
the
test-pre-fork
branch
from
5988e97 to
cb3d2cb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR tests that CI passes functional tests for all code running before the FCMP++ fork. Doesn't need to be merged, just can run against CI as we make changes to fcmp++-stage.