feat: various notification / statistics and performance improvements (#157)

* Fix missing statistics updates for filter matches and relevant transactions

This commit fixes two statistics tracking issues that were accidentally
omitted during the wallet integration refactoring:

1. filters_matched statistic was not being incremented when filter matches
   were detected in sequential sync (sequential/mod.rs:1390-1398)

2. blocks_with_relevant_transactions statistic was not being incremented
   when the wallet found relevant transactions in blocks (block_processor.rs:238-242)

Changes:
- Added stats field to SequentialSyncManager struct to enable statistics
  updates during filter matching
- Updated SequentialSyncManager::new() to accept stats parameter
- Added filters_matched increment when filter match is detected
- Added blocks_with_relevant_transactions increment when wallet processes
  relevant transactions
- Updated DashSpvClient to pass stats when creating SequentialSyncManager

Root cause: These statistics were tracked in old commented-out code
(marked with "TODO: Re-implement with wallet integration") but were never
re-added to the new simplified wallet integration code path.

Result: Statistics now correctly show non-zero values for "Filters Matched"
and "Blocks w/ Relevant Txs" in sync status logs, providing accurate
visibility into sync progress.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* Add TransactionDetected event emission in block processor

- Emit individual TransactionDetected events for each relevant transaction found in blocks
- Include transaction details: txid, confirmed status, block height, and amount
- Remove excessive debug logging from FFI layer

This enables real-time transaction notifications to Swift layer for wallet balance updates.

* Fix account balance not being updated after transaction processing

Root cause: When processing transactions, UTXOs were correctly added/removed
from accounts, but the account's balance field was never recalculated. This
caused wallet-level balance to be correct (calculated by summing account UTXOs)
while individual account balances remained at zero.

Changes:
- Add balance recalculation after UTXO updates in wallet_checker.rs
- Calculate confirmed/unconfirmed/locked balance from account's UTXO set
- Call account.update_balance() with recalculated values

Impact:
- managed_account_get_balance() FFI now returns correct values
- Multi-account wallets will now show correct per-account balances
- Removes need for Swift workaround of using wallet-level balance

Fixes account balance query returning zero despite correct UTXO state.

* Add support for querying all addresses with (0, 0) range

Allow address_pool_get_addresses_in_range to return all addresses when
called with start_index=0 and end_index=0. This provides a clean way to
retrieve all generated addresses without hardcoding a limit.

Implementation:
- Special case: when start=0 and end=0, iterate from 0 to highest_generated
- Normal case: continue with existing range validation and iteration
- Returns all addresses that exist in the pool, including gap limit buffer

This avoids hardcoding arbitrary limits (like 10000) in client code.

* Add FFI transaction list support for managed accounts

Implement transaction retrieval functionality in key-wallet-ffi to expose
transaction data from managed accounts to client applications.

- Add FFITransactionRecord struct with transaction details (txid, amount, height, etc.)
- Implement managed_account_get_transactions() to retrieve transaction array
- Implement managed_account_free_transactions() for proper memory cleanup
- Update C header with new FFI structures and functions

This enables client applications to display wallet transaction history
with complete transaction information including confirmations, fees,
and transaction types.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* perf(dash-spv): optimize header sync and add parallel CFHeaders flow control

- Add CachedHeader wrapper to eliminate redundant X11 hash computations
  during header sync (4-6x reduction in hash operations per header)
- Implement parallel CFHeaders synchronization with flow control, including
  request buffering, sequential processing, retry logic, and timeout handling
- Optimize storage operations by accepting precomputed hashes in
  store_headers_internal to avoid redundant hash computations
- Reduce disk storage index save frequency to every 10k entries instead of
  every periodic save to minimize expensive cloning and serialization
- Use cached hashes throughout validation and storage pipeline in both
  headers.rs and headers_with_reorg.rs
- Add configuration options for CFHeaders flow control with sensible defaults:
  max_concurrent_cfheaders_requests_parallel (50), cfheaders_request_timeout (30s),
  max_cfheaders_retries (3), enable_cfheaders_flow_control (true)
- Remove unused mutable variable in block_processor.rs

These optimizations significantly reduce CPU usage during header sync by
avoiding expensive X11 hash recomputation and enable faster filter header
sync through parallel requests with proper flow control.

* feat(dash-spv): enhance masternode state persistence and synchronization

- Implemented storage of masternode state after sync completion to allow phase manager to detect sync status.
- Updated masternode synchronization logic to use the latest persisted state height for requesting masternode list updates, ensuring accurate base height for ChainLock validation.
- Added logging for potential issues during state persistence and retrieval, improving observability of the synchronization process.

These changes improve the reliability of masternode synchronization and enhance the overall flow control in the dash-spv module.

* docs: update FFI API documentation

* fix: replace Layout::array unwrap with proper error handling in managed_account

Replace the panic-prone unwrap() call on Layout::array with proper error
handling. Now returns false on layout computation failure instead of
panicking at the FFI boundary, improving robustness when handling edge
cases like integer overflow.

* fix: suppress deprecated GenericArray warnings in BIP38 code

Add #[allow(deprecated)] attributes to aes_encrypt and aes_decrypt
functions to suppress CI clippy errors about generic-array 0.14.x
deprecation. The aes crate still requires the old version, making
an upgrade infeasible at this time.

* fix(key-wallet): correct state update flag in transaction routing tests

Fixed two test cases that incorrectly passed Some(&wallet) when
they should have passed None to prevent state updates. The tests
were checking that state is not modified, but were actually
causing updates to occur.

- test_transaction_routing_to_bip32_account: line 167
- test_transaction_affects_multiple_accounts: line 411

All 434 tests now pass.

* fix(dash-spv): add missing CFHeaders flow control fields to test config

Add max_concurrent_cfheaders_requests_parallel, enable_cfheaders_flow_control,
cfheaders_request_timeout_secs, and max_cfheaders_retries fields to the
ClientConfig initialization in network tests to fix compilation error.

* fix(key-wallet-ffi): handle Layout::array overflow safely in deallocator

Replace unwrap() call in managed_account_free_transactions with proper
Result matching. This mirrors the allocation path and prevents potential
panics in FFI context when layout calculation would overflow.

- Match on Layout::array Result instead of unwrap
- Return early if Err to avoid deallocation with invalid layout
- Ensures FFI deallocator is safe and never panics

* refactor: add explicit up-to-date check before computing start_height in CFHeaders request

Skip redundant CFHeaders requests when filter_tip >= stop_height by checking
the up-to-date condition before computing start_height. This avoids unnecessary
height calculations and network requests when the filter is already synced to or
past the target height.

* fix(spv): correct TransactionDetected net_amount via wallet transaction_effect; include affected addresses\n\nfeat(wallet-manager): add WalletInterface::transaction_effect and implement in WalletManager using TransactionRouter and per-account checks\n\ntest(spv): add tests for wallet-provided net, fallback behavior, and negative net with duplicates

* fix: resolve type_complexity clippy warning in block_processor_test

Factor out complex type Arc<Mutex<BTreeMap<Txid, (i64, Vec<String>)>>>
into a type alias TransactionEffectsMap for improved code clarity and
to satisfy clippy type_complexity lint.

---------

Co-authored-by: Claude <[email protected]>

Author: PastaPastaPasta

dash-spv/src/client/block_processor.rs

@@ -234,6 +234,39 @@ impl<W: WalletInterface + Send + Sync + 'static, S: StorageManager + Send + Sync
                 block_hash,
                 height
             );
+
+            // Update statistics for blocks with relevant transactions
+            {
+                let mut stats = self.stats.write().await;
+                stats.blocks_with_relevant_transactions += 1;
+            }
+
+            // Emit TransactionDetected events for each relevant transaction
+            for txid in &txids {
+                if let Some(tx) = block.txdata.iter().find(|t| &t.txid() == txid) {
+                    // Ask the wallet for the precise effect of this transaction
+                    let effect = wallet.transaction_effect(tx, self.network).await;
+                    if let Some((net_amount, affected_addresses)) = effect {
+                        tracing::info!("📤 Emitting TransactionDetected event for {}", txid);
+                        let _ = self.event_tx.send(SpvEvent::TransactionDetected {
+                            txid: txid.to_string(),
+                            confirmed: true,
+                            block_height: Some(height),
+                            amount: net_amount,
+                            addresses: affected_addresses,
+                        });
+                    } else {
+                        // Fallback: emit event with zero and no addresses if wallet could not compute
+                        let _ = self.event_tx.send(SpvEvent::TransactionDetected {
+                            txid: txid.to_string(),
+                            confirmed: true,
+                            block_height: Some(height),
+                            amount: 0,
+                            addresses: Vec::new(),
+                        });
+                    }
+                }
+            }
         }
         drop(wallet); // Release lock
 

dash-spv/src/client/block_processor_test.rs

@@ -12,19 +12,31 @@ mod tests {
     use std::sync::Arc;
     use tokio::sync::{mpsc, oneshot, Mutex, RwLock};
 
+    // Type alias for transaction effects map
+    type TransactionEffectsMap =
+        Arc<Mutex<std::collections::BTreeMap<dashcore::Txid, (i64, Vec<String>)>>>;
+
     // Mock WalletInterface implementation for testing
     struct MockWallet {
         processed_blocks: Arc<Mutex<Vec<(dashcore::BlockHash, u32)>>>,
         processed_transactions: Arc<Mutex<Vec<dashcore::Txid>>>,
+        // Map txid -> (net_amount, addresses)
+        effects: TransactionEffectsMap,
     }
 
     impl MockWallet {
         fn new() -> Self {
             Self {
                 processed_blocks: Arc::new(Mutex::new(Vec::new())),
                 processed_transactions: Arc::new(Mutex::new(Vec::new())),
+                effects: Arc::new(Mutex::new(std::collections::BTreeMap::new())),
             }
         }
+
+        async fn set_effect(&self, txid: dashcore::Txid, net: i64, addresses: Vec<String>) {
+            let mut map = self.effects.lock().await;
+            map.insert(txid, (net, addresses));
+        }
     }
 
     #[async_trait::async_trait]
@@ -64,6 +76,15 @@ mod tests {
         async fn describe(&self, _network: Network) -> String {
             "MockWallet (test implementation)".to_string()
         }
+
+        async fn transaction_effect(
+            &self,
+            tx: &Transaction,
+            _network: Network,
+        ) -> Option<(i64, Vec<String>)> {
+            let map = self.effects.lock().await;
+            map.get(&tx.txid()).cloned()
+        }
     }
 
     fn create_test_block(network: Network) -> Block {
@@ -110,6 +131,15 @@ mod tests {
 
         // Send block processing task
         let (response_tx, _response_rx) = oneshot::channel();
+
+        // Prime wallet with an effect for the coinbase tx in the genesis block
+        let txid = block.txdata[0].txid();
+        {
+            let wallet_guard = wallet.read().await;
+            wallet_guard
+                .set_effect(txid, 1234, vec!["XyTestAddr1".to_string(), "XyTestAddr2".to_string()])
+                .await;
+        }
         task_tx
             .send(BlockProcessingTask::ProcessBlock {
                 block: Box::new(block.clone()),
@@ -120,22 +150,47 @@ mod tests {
         // Process the block in a separate task
         let processor_handle = tokio::spawn(async move { processor.run().await });
 
-        // Wait for event
+        // Wait for events; capture the TransactionDetected for our tx
+        let mut saw_tx_event = false;
         tokio::time::timeout(std::time::Duration::from_millis(100), async {
             while let Some(event) = event_rx.recv().await {
-                if let SpvEvent::BlockProcessed {
-                    hash,
-                    ..
-                } = event
-                {
-                    assert_eq!(hash.to_string(), block_hash.to_string());
-                    break;
+                match event {
+                    SpvEvent::TransactionDetected {
+                        txid: tid,
+                        amount,
+                        addresses,
+                        confirmed,
+                        block_height,
+                    } => {
+                        // Should use wallet-provided values
+                        assert_eq!(tid, txid.to_string());
+                        assert_eq!(amount, 1234);
+                        assert_eq!(
+                            addresses,
+                            vec!["XyTestAddr1".to_string(), "XyTestAddr2".to_string()]
+                        );
+                        assert!(confirmed);
+                        assert_eq!(block_height, Some(0));
+                        saw_tx_event = true;
+                    }
+                    SpvEvent::BlockProcessed {
+                        hash,
+                        ..
+                    } => {
+                        assert_eq!(hash.to_string(), block_hash.to_string());
+                        if saw_tx_event {
+                            break;
+                        }
+                    }
+                    _ => {}
                 }
             }
         })
         .await
         .expect("Should receive block processed event");
 
+        assert!(saw_tx_event, "Should emit TransactionDetected with wallet-provided effect");
+
         // Verify wallet was called
         {
             let wallet = wallet.read().await;
@@ -292,6 +347,158 @@ mod tests {
         let _ = processor_handle.await;
     }
 
+    #[tokio::test]
+    async fn test_transaction_detected_fallback_when_no_wallet_effect() {
+        let (processor, task_tx, mut event_rx, _wallet, storage) = setup_processor().await;
+
+        // Create a test block
+        let block = create_test_block(Network::Dash);
+        let block_hash = block.block_hash();
+        let txid = block.txdata[0].txid();
+
+        // Store header so height lookup succeeds
+        {
+            let mut storage = storage.lock().await;
+            storage.store_headers(&[block.header]).await.unwrap();
+        }
+
+        // Send block processing task without priming any effect (transaction_effect will return None)
+        let (response_tx, _response_rx) = oneshot::channel();
+        task_tx
+            .send(BlockProcessingTask::ProcessBlock {
+                block: Box::new(block.clone()),
+                response_tx,
+            })
+            .unwrap();
+
+        // Process
+        let processor_handle = tokio::spawn(async move { processor.run().await });
+
+        let mut saw_tx_event = false;
+        tokio::time::timeout(std::time::Duration::from_millis(100), async {
+            while let Some(event) = event_rx.recv().await {
+                match event {
+                    SpvEvent::TransactionDetected {
+                        txid: tid,
+                        amount,
+                        addresses,
+                        confirmed,
+                        block_height,
+                    } => {
+                        assert_eq!(tid, txid.to_string());
+                        assert_eq!(
+                            amount, 0,
+                            "fallback amount should be 0 when no effect available"
+                        );
+                        assert!(addresses.is_empty(), "fallback addresses should be empty");
+                        assert!(confirmed);
+                        assert_eq!(block_height, Some(0));
+                        saw_tx_event = true;
+                    }
+                    SpvEvent::BlockProcessed {
+                        hash,
+                        ..
+                    } => {
+                        assert_eq!(hash.to_string(), block_hash.to_string());
+                        if saw_tx_event {
+                            break;
+                        }
+                    }
+                    _ => {}
+                }
+            }
+        })
+        .await
+        .expect("Should receive events");
+
+        assert!(saw_tx_event, "Should emit TransactionDetected with fallback values");
+
+        // Shutdown
+        drop(task_tx);
+        let _ = processor_handle.await;
+    }
+
+    #[tokio::test]
+    async fn test_transaction_detected_negative_amount_and_duplicate_addresses() {
+        let (processor, task_tx, mut event_rx, wallet, storage) = setup_processor().await;
+
+        // Create a test block
+        let block = create_test_block(Network::Dash);
+        let block_hash = block.block_hash();
+        let txid = block.txdata[0].txid();
+
+        // Store header so height lookup succeeds
+        {
+            let mut storage = storage.lock().await;
+            storage.store_headers(&[block.header]).await.unwrap();
+        }
+
+        // Prime wallet with negative amount and duplicate addresses
+        {
+            let wallet_guard = wallet.read().await;
+            wallet_guard
+                .set_effect(
+                    txid,
+                    -500,
+                    vec!["DupAddr".to_string(), "DupAddr".to_string(), "UniqueAddr".to_string()],
+                )
+                .await;
+        }
+
+        // Send block processing task
+        let (response_tx, _response_rx) = oneshot::channel();
+        task_tx
+            .send(BlockProcessingTask::ProcessBlock {
+                block: Box::new(block.clone()),
+                response_tx,
+            })
+            .unwrap();
+
+        // Process
+        let processor_handle = tokio::spawn(async move { processor.run().await });
+
+        let mut saw_tx_event = false;
+        tokio::time::timeout(std::time::Duration::from_millis(100), async {
+            while let Some(event) = event_rx.recv().await {
+                match event {
+                    SpvEvent::TransactionDetected {
+                        txid: tid,
+                        amount,
+                        addresses,
+                        confirmed,
+                        block_height,
+                    } => {
+                        assert_eq!(tid, txid.to_string());
+                        assert_eq!(amount, -500);
+                        // BlockProcessor uses wallet-provided addresses as-is (no dedup here)
+                        assert_eq!(addresses, vec!["DupAddr", "DupAddr", "UniqueAddr"]);
+                        assert!(confirmed);
+                        assert_eq!(block_height, Some(0));
+                        saw_tx_event = true;
+                    }
+                    SpvEvent::BlockProcessed {
+                        hash,
+                        ..
+                    } => {
+                        assert_eq!(hash.to_string(), block_hash.to_string());
+                        if saw_tx_event {
+                            break;
+                        }
+                    }
+                    _ => {}
+                }
+            }
+        })
+        .await
+        .expect("Should receive events");
+
+        assert!(saw_tx_event, "Should emit TransactionDetected with negative net and duplicates");
+
+        // Shutdown
+        drop(task_tx);
+        let _ = processor_handle.await;
+    }
+
     #[tokio::test]
     async fn test_process_mempool_transaction() {
         let (processor, task_tx, _event_rx, wallet, _storage) = setup_processor().await;

dash-spv/src/client/config.rs

@@ -163,6 +163,19 @@ pub struct ClientConfig {
     /// Rate limit for CF header requests per second (default: 10.0).
     pub cfheaders_request_rate_limit: Option<f64>,
 
+    // CFHeaders flow control configuration
+    /// Maximum concurrent CFHeaders requests for parallel sync (default: 50).
+    pub max_concurrent_cfheaders_requests_parallel: usize,
+
+    /// Enable flow control for CFHeaders requests (default: true).
+    pub enable_cfheaders_flow_control: bool,
+
+    /// Timeout for CFHeaders requests in seconds (default: 30).
+    pub cfheaders_request_timeout_secs: u64,
+
+    /// Maximum retry attempts for failed CFHeaders batches (default: 3).
+    pub max_cfheaders_retries: u32,
+
     /// Rate limit for filter requests per second (default: 50.0).
     pub filters_request_rate_limit: Option<f64>,
 
@@ -238,6 +251,11 @@ impl Default for ClientConfig {
             blocks_request_rate_limit: None,
             start_from_height: None,
             wallet_creation_time: None,
+            // CFHeaders flow control defaults
+            max_concurrent_cfheaders_requests_parallel: 50,
+            enable_cfheaders_flow_control: true,
+            cfheaders_request_timeout_secs: 30,
+            max_cfheaders_retries: 3,
             // QRInfo defaults (simplified per plan)
             qr_info_extra_share: false, // Matches DMLviewer.patch default
             qr_info_timeout: Duration::from_secs(30),

dash-spv/src/client/mod.rs

@@ -311,6 +311,7 @@ impl<
             received_filter_heights,
             wallet.clone(),
             state.clone(),
+            stats.clone(),
         )
         .map_err(SpvError::Sync)?;
 

dash-spv/src/network/tests.rs

@@ -146,6 +146,10 @@ mod multi_peer_tests {
             enable_filter_flow_control: true,
             filter_request_delay_ms: 0,
             max_concurrent_filter_requests: 50,
+            max_concurrent_cfheaders_requests_parallel: 50,
+            enable_cfheaders_flow_control: true,
+            cfheaders_request_timeout_secs: 30,
+            max_cfheaders_retries: 3,
             enable_cfheader_gap_restart: true,
             cfheader_gap_check_interval_secs: 15,
             cfheader_gap_restart_cooldown_secs: 30,