Get multisig working pre-FCMP++/Carrot fork#291
Draft
j-berman wants to merge 76 commits intoseraphis-migration:fcmp++-stagefrom
Draft
Get multisig working pre-FCMP++/Carrot fork#291j-berman wants to merge 76 commits intoseraphis-migration:fcmp++-stagefrom
j-berman wants to merge 76 commits intoseraphis-migration:fcmp++-stagefrom
Conversation
This replaces `ver_rct_non_semantics_simple_cached()` with an API that offloads the responsibility of tracking input verification successes to the caller. The main caller of this function in the codebase, `cryptonote::Blockchain()` instead keeps track of the verification results for transaction in the mempool by storing a "verification ID" in the mempool metadata table (with `txpool_tx_meta_t`). This has several benefits, including: * When the mempool is large (>8192 txs), we no longer experience cache misses and unnecessarily re-verify ring signatures. This greatly improves block propagation time for FCMP++ blocks under load * For the same reason, reorg handling can be sped up by storing verification IDs of transactions popped from the chain * Speeds up re-validating every mempool transaction on fork change (monerod revalidates the whole tx-pool on HFs monero-project#10142) * Caches results for every single type of Monero transaction, not just latest RCT type * Cache persists over a node restart * Uses 512KiB less RAM (8192*2*32B) * No additional storage or DB migration required since `txpool_tx_meta_t` already had padding allocated * Moves more verification logic out of `cryptonote::Blockchain` Furthermore, this opens the door to future multi-threaded block verification speed-ups. Right now, transactions' input proof verification is limited to one transaction at a time. However, one can imagine a scenario with verification IDs where input proofs are optimistically multi-threaded in advance of block processing. Then, even though ring member fetching and verification is single-threaded inside of `cryptonote::Blockchain::check_tx_inputs()`, the single thread can skip the CPU-intensive cryptographic code if the verification ID allows it. Also changes the default log category in `tx_verification_utils.cpp` from "blockchain" to "verify".
Co-authored-by: j-berman <justinberman@protonmail.com>
Co-authored-by: jeffro256 <jeffro256@tutanota.com> Co-authored-by: Luke Parker <lukeparker5132@gmail.com> Co-authored-by: SyntheticBird45 <someoneelse.is_on.github.rio7x@simplelogin.com>
Otherwise we can end up double counting txs towards the weight, which can over-state the pool weight. E.g. relay tx to node in stem phase, add its weight to pool weight, then receive tx from another node, then bump the pool weight again. That double counts the tx towards the pool weight. If the weight exceeds the max, the node will "prune" txs from the pool. Thus, over-counting is probably a cause of, but perhaps not the only cause of: seraphis-migration#148
Curve Trees: handle get_max_concurrency() == 0
tx pool: only increment m_txpool_weight for newly added pool txs
Fixes pruning the database under FCMP++ and prevents future corruption by checking the version value inside the properties table.
Log total pool weight
…prune blockchain_prune: add FCMP tables and check DB version
…ver-ids Fix FCMP++ batch verification collecting ver ID's
fcmp++ rust: ASAN support
Also add documention.
…arrot_devs carrot_impl: refactor scanning_tools to use Carrot devices
…rint cryptonote_basic: remove BP+ clawback debug print in weight func
…d_designator carrot_impl: specify all fields in aggregate init (1)
wallet: disclude all 0-amount carrot outputs from get_transfers(false)
…_size unit_tests: dump pruned FCMP++ tx byte size table
…zation_string carrot_core: add 'Monero' blake2b personal string
…election_count carrot_impl: fix input selection subroutine, and enable limited inputs
process_new_scanned_transaction perf timer is very noisy
…::view_key_scalar_mult_x25519
…ddr_dev carrot_impl: fix infinite loop in cryptonote_hierarchy_address_device…
txpool: don't drop cnxn that relays high FCMP++ ref block
Reduces logging noise and speeds up some algos
carrot_impl: filter 0-amount inputs before input selection
txpool: don't process txs that would push pool over capacity
- Replaced OutputPairType with the OutputPair variant. - Moved OutputPair into fcmp_pp_types.h and its serialization fn's into fcmp_pp_serialization.h. - Consolidated logic for determining if an output has been/should be checked for torsion, AND the logic for if we should use the biased hash to point for the key image generator, into cryptonote_format_utils. This provides a stronger guarantee we'll correctly update the logic as needed in the future, and keeps the logic organized neatly in one place. - Used inlines in cryptonote_format_utils.h so that the fcmp_pp lib does not have to link cryptonote_basic, triggering a circular dep: cryptonote_basic->ringct_basic->fcmp_pp->cryptonote_basic
Cleaner structure for biased key gen/torsion checking
Unbiased hash-to-point integration
Reuse the background sync cache for multisig sync, so that upon importing multisig info, the wallet doesn't need to rescan the chain and instead can just process the background cache.
j-berman
force-pushed
the
multisig-bg-sync
branch
from
603ed2f to
27db6ba
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reuses the background sync cache for multisig sync, so that upon importing multisig info, the wallet doesn't need to rescan the chain and instead can just process the background cache. This change is upstream-able.
This is a required change for a FCMP++ compatible wallet because with the existing impl (before this PR), the wallet may rescan the chain from many blocks below chain tip. With FCMP++, the wallet can't pop more than n blocks (100 is the default) because the pruned curve tree cannot go back that far, it has a max depth.
I'm planning to look into porting @kayabaNerve's drop-in compatible multisig next, building on top of this draft.