feat: fungibles precompiles (#573)

* build(deps): disable default-features for serde

* feat(api-vnext): new 'pallet' to provide fungible api(s) via precompiles

* feat(devnet): add fungibles precompiles

* build(deps): propagate runtime-benchmarks feature

* build: disable doctest based on incompatible comments in .sol files

* fix(precompiles/interfaces): use calldata specifiers for string
parameters

* style(precompiles/interfaces): formatting

* refactor: use upcoming assetidextractor from pallet-assets

* refactor(api-vnext): resolve clippy warnings

* style: formatting

* feat(pop-api): add initial fungibles api and example

* chore: rename example directory

* docs: remove trailing slash in comment

* refactor(pop-api): improvements to implementation

* test(devnet): improvements to fungibles precompile contract test

* refactor(pop-api): remove redundant code for manual precompile calls

* refactor(fungibles): ensure feature parity with prior implementation

* refactor(pop-api): use explicit sol encoding for precompile contract
refs

* docs(examples): update parameter description based on review comment

* docs(api): add public function doc

* docs(example): update constructor parameter doc style to match messages

* docs(example): use account instead of address in doc comments

* docs(example): add todo prefix to comment

* docs(api): add public function doc

* test(api): remove additional check deemed unnecessary

* docs(api): add fungibles doc comments

* refactor(precompiles): add license and version requirement

* refactor(precompiles): update return types after rebase

* test(api): resolve breaking changes after rebase

* test(api): improve fungibles precompile tests after rebase

* test(api): improve erc20 precompile tests after rebase

* test(api): define precompile constants within mock runtime

* refactor(api): use asset id extractors from pallet-assets

* test(devnet): resolve breaking changes after rebase

* refactor(api): import soltype

* refactor(api): correct fungibles trait parameter names

* refactor(devnet): simplify selector implementations

* test(api): add initial integration tests

* ci: add job for pop-api-vnext integration tests

* test(api): minimal implementation of transfer test

Solidity error handling still unsupported.

* ci: set rustflags for pop-api-vnext integration tests

* test(runtime): disable precompiles with contract test

Disabled until it can be replaced completely by integration tests.

* test(api): add transfer_from test

* ci: exclude pop-api-integration-tests from test and coverage jobs

* test(api): add approve test

* test(api): add contract.account_id() helper

* test(api): add increase/decrease allowance tests

* refactor(api): separate out fungibles errors module

* feat(api): add fungibles events

* test(api): add create test

* test(api): add start_destroy test

* test(api): make contract instantiation origin explicit to simplify api

* test(api): add set_metadata test

* test(api): update expectation message to .polkavm

* test(api): add clear_metadata test

* test(api): reuse sol types to ease encoding

* test(api): add mint test

* test(api): add burn test

* test(api): add create with instantiation test

* feat(api): initial precompile benchmarking

* refactor(api): charge weight for increase/decrease allowance calls

* refactor(api): improve decrease allowance implementation

* feat(api): add clear_metadata benchmark

* feat(api): add set_metadata benchmark

Also adds a few additional improvements.

* feat(api): add start_destroy benchmark

* feat(api): add create benchmark

* refactor(api): move benchmark assertion into block

* refactor(api): use call_precompile helper

Provides decoding to ease assertions.

* feat(api): add mint benchmark

* feat(api): add burn benchmark

* feat(api): add transfer benchmark

* feat(api): add transfer_from benchmark

* refactor(api/fungibles/benchmarking): add copy trait bound to asset id
to eliminate manual clones

* refactor(api/fungibles): refund appropriate burn weight on failure

* style(api/fungibles): formatting

* refactor(api/fungibles/erc20): charge benchmarked weight

Uses benchmarks from the fungibles precompile which should be exactly
the same for these functions.

* chore(api): add cargo.lock

* test(devnet): set gas limit now that precompiles charge weight

* test(devnet): remove contract and selector integration tests as available elsewhere now

* test(api): add selectors test

* test(api): add initial tests

* test(api): remove unnecessary helper

* test(api): add increase/decrease tests

* test(api): add create test

* test(api): add start_destroy test

* test(api): add set_metadata test

* test(api): add clear_metadata test

* test(api): add mint test

* test(api): add burn test

* test(api): add exists test

* refactor(api): share total_supply implementation

* refactor(api): share balance implementation

* refactor(api): share allowance implementation

* refactor(api): share metadata implementation

* test(api): explicitly endow when more than ed required

* refactor(api): reorganize for versioning

* style: formatting

* build(deps): update ink after sol-abi improvements merged

* ci: explicitly cache pallet-revive-fixtures output

* test(api/integration-tests): fix imports after refactor

* ci: remove cached pallet-revive fixtures

* ci: only list if directory exists

* ci(init): make crate caching configurable

* ci(init): bump rust-cache version

* refactor(pallets/api): address clippy warning

* docs(api): add missing precompile index comment

* refactor(pallets/api): ensure weight charge updated on both approve success and
failure

* refactor(pallets-api): update event handling based on reference
implementation

* docs: remove todo comment

* test(api/integration-tests): constructor encoding

Uses `all` encoding to support multiple constructors after solidity
constructor support added.

* docs(api): fix transfer comment

* docs(api): port readmes

* refactor(pallets/api): simplify u256 conversion

* refactor(pallets/api): relocate benchmarking call setup and resolve
issues

* refactor(api): update ink, eliminating manual encoding/decoding

* fix(api): add missing imports after refactor

* refactor: implement initial precompile reverts

* test(api): add initial error checks

* refactor(api): improved fungibles error handling

* refactor(api): improved erc20 error handling

* build(deps): add missing base64 feature

* refactor: resolve missing imports/trait bounds for runtime-benchmarks

* refactor(pallets/api): resolve clippy warnings

* test(pallets/api): fix minimum balance consideration with transfer

* test(pallets/api): remove unnecessary println

* test(pallets/api): fix minimum balance consideration with transfer
benchmark

* fix(api): allow removal of approval with zero value allowance

* refactor(api): check set_metadata result after refactoring

* chore: update cargo.lock after rebase

* style: formatting after rebase

* chore(api): update cargo.lock for contracts

* docs(api): remove todos

* ci: enable global cache disabling

* ci: reenable crate caching

* docs(pallets/api): remove token id size todo after discussion

* test(api): relocate and reduce initial value, gas and storage limits

* refactor(pallets/api): ensure all errors apart from insufficient gas
result in revert

* refactor(api): ensure full error coverage

* test(api): update expected errors

* chore(api): update cargo.lock

* chore: update cargo.lock

* refactor(pallets/api): simplify origin

* refactor(pallets/api): fix imports

* refactor(api): reuse selectors

* feat: messaging precompiles (#584)

* feat(pallets/api): add messaging precompiles

* feat(devnet): add messaging pallet

* feat(api): add initial messaging apis

* feat(api): add messaging example

* refactor(api): update callback encoding

* build(pallets/api): add empty ismp feature

* refactor: port on_initialize hook implementation

* style: formatting

* build(pallets/api): adjust ismp feature

* feat(pallets/api): add messaging benchmarks

* refactor(ismp): use u64 height

Destination state machine could use larger type, so standardise on the
same type provided by ismp.

* refactor(api): update imports after rebase

* style: formatting after rebase

* fix: remove duplicate imports after rebase

* refactor(api): add initial messaging error handling

* test(api): add error tests

* test(pallets/api): add remove tests

* test(pallets/api): add new_query tests

* test(pallets/api): add call tests

* test(pallets/api): add process_callback_weight tests

* test(pallets/api): add deposit_callback_event tests

* test(pallets/api): add fee management test

* test(pallets/api): add xcm_response tests

* test(pallets/api): add ismp tests

* refactor: limit ismp_response helper to tests

* refactor(pallets/api): use try_convert

* refactor(pallets/api): update messaging weights

* refactor(pallets/api): use global message identifier

Stores contract address in message to ensure authorised removal.
Additional benefits beyond reduced storage requirements are that any
contract can poll and use responses until originator removes.

* refactor(pallets/api): update weights after benchmarking

* fix(pallets/api): replace bounded_vec! usage within benchmarks

* refactor(pallets/api): callback response interfaces and encoding

* refactor(api): precompile and callback trait and type encoding
improvements

* feat(devnet): add messaging precompiles

* test(api): add initial messaging integration tests

* test(api): update ink dependency

* refactor(api): update ink dependency

* test(api): restore onpostresponse handler

* test(pallets/api): fix tests after refactor

* test(api): add ismp post integration tests

* refactor(pallets/api): rename response handler functions due to selector
clashes

A workaround for ink which does not allow two traits to be implemented
with a message with the same resulting selector.

* refactor(api): rename response handler functions due to selector clashes

* test(api): add xcm messaging integration tests

* build(api): updating ink to master

* docs(pallets/api): remove unnecessary todos

* docs(api): remove unnecessary todos

* docs(api): add hashed_account function docs

* feat(pallets/api): add (para) id getter to messaging precompiles

* feat(api): add (para) id getter to apis

* test(api): add (para) id getter integration test

* refactor(api): update messaging example with improvements

* refactor(api): improve return types of xcm execute and send helpers

* refactor(api): use ink_abi cfg for callback encoding

* refactor(pallets/api): restore ismp deposits

* test(pallets/api): increase message deposit

* docs(pallets/api): update comments on necessary clones of bytes type
during abi encoding

Data needs to be encoded for persistence if callback fails and therefore
cannot be owned.

* refactor(pallets/api): better handling of invalid callback encoding

* test(api): apply reduced initial value, gas, storage limits

* fix(pallets/api): surface storage deposit limit into callback
specification

* fix(api): allow specification of storage deposit limit for callback

* test(api): ensure storage deposit limit works

Also adjusts gas limits based on contract changes.

* refactor(devnet): update callbackexecutor.execute implementation to use
storage deposit limit

* refactor(api): update messaging example contract with storage deposit
limit

* fix(pallets/api): allow caller to specify ismp post recipient
module/address

* fix(api): allow caller to specify ismp post recipient module/address

* test(api): specify ismp post recipient mpdule/address

* refactor(devnet): configure max ismp post module/address length as 32
bytes

* docs(pallets/api): remove messageid size todo

u64::MAX provides a sufficiently large number for message identifier and
aligns with the same type used by XCM for query identifiers.

* refactor(pallets/api): add simple on_accept benchmark for consistency,
along with supporting module weight tests

* refactor(devnet): name ismp module via import alias

* refactor(devnet): add ismp weight provider

* docs(pallets/api): remove todo after tests added

* test(pallets/api): add integrity test to ensure maxresponselen

* refactor(pallets/api): update integrity test to include values

* refactor(pallets/api): update tryconvert usages to dispatcherror after
rebase

* chore: update cargo.lock

* chore(api): update cargo.lock for messaging contract

* refactor(pallets/api): use improved origin

* refactor(pallets/api): wrap fallible calls in closure to map errors

* refactor(pallets/api): use try_convert to simplify safe conversion

* refactor(pallets/api): ensure all errors apart from insufficient gas
result in revert

* refactor(api): ensure full error coverage

* test(api): update callback gas limits

* docs(api): add readmes

* docs(api): relocate messaging within readme

* docs(pallet-api): add weights overview from original pr

* docs(pallets-api): relocate weights.md

* docs(pallet-api): add description to messaging module

* docs(pallet-api): add readme

* docs(pallet-api): add initial messaging readme

* docs(pallets/api): add diagrams to messaging readme

---------

Co-authored-by: Frank Bell <[email protected]>

Author: evilrobot-01

.github/actions/init/action.yml

@@ -1,8 +1,19 @@
 name: Initialize
 description: This action initializes a runner for use in other actions.
 inputs:
+  cache:
+    description: Whether to enable Rust caching. Defaults to true.
+    default: "true"
+  cache-all-crates:
+    description: Whether all crates should be cached, or only dependent crates. Defaults to true.
+    default: "true"
+  cache-directories:
+    description: Additional non-workspace directories to be cached, separated by newlines.
+  cache-on-failure:
+    description: Cache even if the build fails. Defaults to false.
+    default: "true"
   cache-key:
-    description: "The key to be used for the cache"
+    description: An additional optional key to be added. Useful for jobs utilizing a matrix.
 
 runs:
   using: "composite"
@@ -25,8 +36,10 @@ runs:
         sudo rm -rf "$AGENT_TOOLSDIRECTORY"
 
     - name: Rust Cache
-      uses: Swatinem/[email protected]
+      if: ${{ inputs.cache == 'true' }}
+      uses: Swatinem/[email protected]
       with:
-        cache-on-failure: true
-        cache-all-crates: true
-        key: ${{ inputs.cache-key }}
+        cache-all-crates: ${{ inputs.cache-all-crates }}
+        cache-directories: ${{ inputs.cache-directories }}
+        cache-on-failure: ${{ inputs.cache-on-failure }}
+        key: ${{ inputs.cache-key }}

.github/workflows/ci.yml

@@ -2,9 +2,9 @@ name: ci
 
 on:
   push:
-    branches: [ main ]
+    branches: [main]
   pull_request:
-    types: [ opened, synchronize, reopened, ready_for_review ]
+    types: [opened, synchronize, reopened, ready_for_review]
 
 concurrency:
   # Cancel any in-progress jobs for the same pull request
@@ -61,7 +61,9 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
+
       - uses: "./.github/actions/init"
+
       - name: Check Build with ISMP
         run: |
           cargo check --release --locked --workspace --exclude integration-tests --features=ismp,runtime-benchmarks,try-runtime
@@ -113,7 +115,7 @@ jobs:
       - uses: "./.github/actions/init"
 
       - name: Run tests
-        run: cargo test --release --locked --workspace --exclude integration-tests --exclude pop-api-integration-tests --features=runtime-benchmarks
+        run: cargo test --release --locked --workspace --exclude integration-tests --exclude pop-api-integration-tests --exclude pop-api-vnext-integration-tests --features=runtime-benchmarks
 
   integration-tests:
     needs: lint
@@ -150,6 +152,19 @@ jobs:
       - name: Run API integration tests
         run: cargo test -p pop-api-integration-tests --release --locked --features "${{ matrix.features }}"
 
+  api-vnext-integration-tests:
+    needs: lint
+    runs-on: ubuntu-latest
+    env:
+      RUSTFLAGS: '--cfg ink_abi="all"'
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: "./.github/actions/init"
+
+      - name: Run API integration tests
+        run: cargo test -p pop-api-vnext-integration-tests --release --locked
+
   api-tests:
     runs-on: ubuntu-latest
     steps:
@@ -194,7 +209,7 @@ jobs:
         uses: taiki-e/install-action@cargo-llvm-cov
 
       - name: Generate code coverage
-        run: cargo llvm-cov --features=runtime-benchmarks,try-runtime,ismp --workspace --exclude integration-tests --exclude pop-api-integration-tests --lib --bins --codecov --output-path codecov.json
+        run: cargo llvm-cov --features=runtime-benchmarks,try-runtime,ismp --workspace --exclude integration-tests --exclude pop-api-integration-tests --exclude pop-api-vnext-integration-tests --lib --bins --codecov --output-path codecov.json
 
       - name: Upload to codecov.io
         uses: codecov/codecov-action@v4