-
Notifications
You must be signed in to change notification settings - Fork 40
Wallet server update #845
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Wallet server update #845
Conversation
* feat(json_rpc): Add `RestoreMembershipProof` call * feat(json_rpc): `GetBlocks` call on Wallet and prepare types for `submitTransaction` * style(json_rpc): Simplify error conversion from RpcError to JsonError * feat(json_rpc): Add `SubmitTransaction` call and `SubmitTx` to channel * feat(json_rpc): Test an off-node wallet scenario and refactor some RPC types * feat(json_rpc): Expose appendix on BlockKernel * test(json_rpc): Cleanup and use deterministic timestamps on off_node test * feat(json_rpc): Avoid TransactionKernelProxy impl and ProofCollection-capability on tests
Because of the added announcements, transparent transactions require a higher transaction fee than non-transparent, regular transactions.
Also, slightly clean up inline stack annotations by removing irrelevant entries, and inline some used-once TASM. changelog: ignore
Potential DoS vector. Change applied to `peer_loop`, not `is_valid`. Credit to AllFather team.
Prevents short malicious messages from allocating a lot of memory. Credit to AllFather team.
Countermeasure against semaphore starvation attack. Credit to AllFather team.
Potential DoS vector. Credit to AllFather team.
Const `MAX_ANNOUNCEMENT_MESSAGE_SIZE` is does not contribute to block (in)validity, but is enfored at the peer-to-peer level as a transaction relay and mempool insertion policy. This commit moves the const definition to the file where it is used, `peer_loop.rs` and clarifies its purpose in the docstring.
…us const Const `MAX_BLOCK_SIZE_IN_FORK_RECONCILIATION` is enforced in the course of fork reconciliation. Previously it was set to 10 MB, now it is set to 2x 8 MB, where the 8 MB comes from the block size. Also: clarify purpose.
There is already a memory limit being enforced implicitly, namely through: 1. Requiring that every block have a size less than or equal to `MAX_BLOCK_SIZE_IN_FORK_RECONCILIATION`. 2. Requiring that the number of blocks being reconciled does not exceed `self.global_state_lock.cli().sync_mode_threshold`. This commit removes the memory limit in `try_ensure_path` as well as the state variable used to track memory use for fork reconciliation.
Instead of using protocol agreement in the calculation of `PeerScore`, we reject any connections if the two peers do not agree on the protocol. Co-authored-by: Alan Szepieniec <alan@neptune.cash>
1. Upgrade dependency tasm-lib to v2.0.0 to get the Triton VM v2.0.0 which fixes a soundness error in Triton VM's proof system. 2. `block-claims` Add a new binary `block-claims` that reads the archival block database and outputs a list of block validity claims for blocks 0..=arg. This binary was used for (3). 3. `checkpoint.dat` Add new file `assets/checkpoint.dat` which contains the block validity claims for blocks 0..=23400. 4. Accept checkpoint Call and add function in archival state initializer, called `accept_checkpoint`, which reads static program memory macroically guaranteed to agree with `checkpoint.dat`, parses it as a list of claims, and populates the true claims cache with these claims. 5. Untest true claims cache Up until now, the true claims cache was used only for tests. Going forward it is going to be used in production as well, so remove the test flag. 6. Style: IntoIterator Modify `cache_true_claim` --> `cache_true_claims` to take a list (technically, a `IntoIterator<_>` object) of `Claim`s rather than a single Claim. Avoid taking the lock every time when there are some 23401 claims to cache, in favor of taking it only once. 7. `pub` Mark `pub` functions, structs, and fields that are used in binary `block-claims`. 8. Define consensus rule set for sound version of Triton VM test: Use tvm proof version 1 for all tests This is done to prevent the need of two different versions of Triton VM. With this change only version 1, which is assumed to be sound, is used. So version 0 is avoided since it is unsound and we can avoid its dependency in neptune-core. This ensures that production and test behavior are somewhat similar: they only deviate on Triton VM versions, not on any other consensus logic (except for size of memory requirements in the PoW). This means that existing tests verifying transitions across hardfork boundaries can keep working and keep being meaningful. Example: The test in `peer_loop`: `hardfork_alpha_happy_case`. Introduces a new `enum` `TritonProofVersion` such the the distiction on the upstream Triton VM version can be better isolated. Co-authored-by: Alan Szepieniec <alan@neptune.cash>
Previously, sync blocks (individual blocks requested by height, typically by the sync loop) were forwarded to the sync loop with zero validation, exposing the node to a potential denial-of-service attack. This commit makes that attack more expensive: now the incoming block's proof is validated, so the attacker does still need to include valid block proofs (and appendices). It is not possible to validate the entire block because that relation pertains to the block's predecessor which is not available in this context. Co-authored-by: Alan Szepieniec <alan@neptune.cash>
Co-authored-by: Alan Szepieniec <alan@neptune.cash>
…testnet This allows us to test that the recently implemented upgrade of the Triton VM version to proof version 1 works on testnet also, that the transition from proof version 0 to proof version 1 is smooth. Co-authored-by: Alan Szepieniec <alan@neptune.cash>
Co-authored-by: Alan Szepieniec <alan@neptune.cash>
Co-authored-by: Alan Szepieniec <alan@neptune.cash>
On startup, check the block program proof for the current tip. If it does not pass validation, then roll back to the checkpoint. Ensure we don't panic if rollback luca block is not known; handle rollback of block correctly in case the threshold/luca block, the latest block valid under the old version of Triton VM, is not known. Co-authored-by: Thorkil Schmidiger <thor@neptune.cash>
Set correct consensus rule set when creating the coinbase transaction. Prior to this commit, there was an off-by-one error here which had never been caught because the consensus rule set for transactions (and even that of block validity) had never changed through a hardfork. Co-authored-by: Alan Szepieniec <alan@neptune.cash>
Since the "true claims" is populated with only canonical blocks prior to block height 23401, block imports from a downloaded list of blocks cannot handle reorganizations, orphaned blocks, in these old blocks, unless we simply choose to ignore invalid blocks prior to this block height. Since the invalid blocks are never applied to the state update function and since we now that they are not canonical, this is fine.
By deleting this env varibale in Makefile, the commands run through
`make` now use the same target directory as those run through `cargo
<{build,test,clippy,fmt,...}>` commands. This saves disk space and
compilation time for users that use both ways of
compiling/testing/linting.
For some reason, leveldb-sys is failing to build with gcc/gcc++. This happens because the cmake build script used by this crate sets the flag '-Wthread-safety' which gcc does not know. Resulting in this error: `c++: error: unrecognized command-line option '-Wthread-safety'` A fix is to switch to clang and download a package that contains a binary for `lstdc++`, otherwise linking in clang fails with the error `"/usr/bin/ld: cannot find -lstdc++: No such file or directory"`. build(Makefile): Verify C++ toolchain before compilation Add two recipes to the Makefile, one for checking presence of clang/clang++, and one for installing it (if on Ubuntu/Debian) or exiting gracefully. Simultaneously with clang/clang++, libstdc++14-dev is installed as well, but presumably any version of C++ can work. Co-authored-by: Alan Szepieniec <alan@neptune.cash>
ratatui 0.29 depended on an unsafe version of lru. Cf.: https://github.com/Neptune-Crypto/neptune-core/security/dependabot/14 and https://cwe.mitre.org/data/definitions/476.html From dependabot: Package lru (Rust) Affected versions >= 0.9.0, < 0.16.3 Patched version 0.16.3
`needless_if` was renamed to `needless_ifs`.
Upgrade a few dependencies such that the code base only depends on one version of itertools, not three as was previously the case.
Add some debug log messages when a new block is applied to the state. This allows us to pinpoint which parts of the state update that is slow. I know that it's the wallet that's currently causing problems for nodes that manage many UTXOs, but I want more information about whether it's the application of a block or the restoration of MS membership proofs (likely the culprit).
Co-authored-by: Alan Szepieniec <alan@neptune.cash>
Ensure that legacy peer-to-peer stack agrees with libp2p stack on version compatibility check. Don't allow connections between v0.5.0 and v0.6.0. Also: Fix bug shared handshake validator, on `HandshakeData`, relating to versions. Co-authored-by: Alan Szepieniec <alan@neptune.cash>
Co-authored-by: Alan Szepieniec <alan@neptune.cash>
Avoid wasting time next time to find bumpable versions. Co-authored-by: Alan Szepieniec <alan@neptune.cash>
Co-authored-by: Alan Szepieniec <alan@neptune.cash>
|
Happy to merge whatever you need onto this branch. But don't you want to switch to the standard JSON/HTTP rpc server that's hosted on port 9797? Doesn't it have all the endpoints you need? If you make this switch, you don't have to maintain your own branch anymore, as you can just rely on |
|
I was focusing on other things; this pr is just to keep my wallet active. i see 90698b6 adds rpc for wallets |
83b82a7
into
Neptune-Crypto:wallet-server
|
Squashed and merged. Hope that's OK. That was the only way to avoid conflicts. I didn't work with a regular merge. |
merged master
this pr changes wallet-server json_rpc endpoint from 9800 to 9802