Commit 279c37f
authored
Perf/ibd speedups (#277)
* Silence per-block LogPrintf calls on sync hot path
Four unconditional LogPrintf calls were firing on every PON header and
every block connect/disconnect during IBD, dominating debug.log volume
and measurably slowing sync. Convert them to LogPrint with a category
so they only fire under -debug=pon or -debug=fluxnode.
- src/pon/pon.cpp: PON difficulty adjustment traces (2x per PON header)
- src/fluxnode/fluxnode.cpp: FluxnodeCache::LogDebugData, plus an
early-return so the per-block string concatenation over
mapStartTxTracker / mapStartTxDOSTracker is skipped when the fluxnode
category isn't active.
* Adopt Bitcoin Core's dbcache split to favor the in-memory UTXO set
Fluxd used to allocate 3/4 of -dbcache to the block tree DB whenever
-insightexplorer was enabled, which left only ~76 MiB of in-memory UTXO
cache at the default -dbcache=450. During IBD this caused frequent
full-cache flushes (multi-second stalls every few thousand blocks).
Switch to Bitcoin Core's allocation: block tree DB gets 1/8 of the
total, capped at 2 MiB without txindex or 1024 MiB with txindex /
insightexplorer. Coin DB cache is capped at 8 MiB. The remainder goes
to the in-memory UTXO set.
At -dbcache=450 with all indexes on, the UTXO cache grows from ~76 MiB
to ~386 MiB; at -dbcache=2000 it reaches ~1.7 GiB, which eliminates
most flush stalls during IBD.
* Implement BIP 152 low-bandwidth mode (peer-initiated compact blocks)
PR #266 shipped high-bandwidth compact block relay (unsolicited
cmpctblock messages to up to three peers). Low-bandwidth mode lets a
peer request a compact block on demand via getdata, saving bandwidth
for edge peers (light clients, metered connections) that haven't opted
into high-bandwidth announcements.
Adds MSG_CMPCT_BLOCK as a new inv type, handled by ProcessGetData.
When a block is within MAX_CMPCTBLOCK_DEPTH (5 blocks) of the tip, the
server responds with a cmpctblock message; older blocks fall back to
sending a full block because the requester is unlikely to have the
mempool state needed to reconstruct it.
Type number 11 is used instead of the BIP 152 canonical value of 4
because type 4 is already allocated to "spork" on the Flux network.
IsFluxnodeType() is tightened to an explicit [4, 10] range so the new
type isn't misrouted.
* Early-return CheckForExpired/UndoExpiredStartTx when tracker is empty
CheckForExpiredStartTx and CheckForUndoExpiredStartTx are called on
every ConnectBlock/DisconnectBlock. For most of chain history the
relevant trackers (mapStartTxTracker, mapStartTxDOSTracker) are empty,
so the iterate-and-conditionally-copy loops do no work but still pay
for the cs_main-adjacent lock acquisition and the subsequent log
lines. Skip the body entirely when there is nothing to do.
The emptiness check runs under the lock, so it races correctly with
concurrent modifications.
* Cap debug.log size and enforce it at runtime
Previously ShrinkDebugFile() only ran at startup and only when -debug
was not set (the -shrinkdebugfile default was !fDebug). With debug=1
the log could grow without bound.
Changes:
- -maxdebugfilesize upper cap is now 10 GiB when -debug is enabled
(so a long debug session has room to breathe) and 2 GiB otherwise.
The 500 MB default is unchanged.
- Default -shrinkdebugfile to true so the cap is enforced even when
debug logging is on.
- Make ShrinkDebugFile() safe to call at runtime by writing the kept
tail to a temp file and atomically renaming; concurrent LogPrintStr
writes land on the unlinked inode and fReopenDebugLog is set so the
next log write reopens on the new file.
- Schedule ShrinkDebugFile() every 5 minutes so a long-running node
with heavy debug output doesn't fill the disk between restarts.
* Harden BIP 152 compact block handling
Two fixes identified by an audit of fluxd's BIP 152 implementation
against Bitcoin Core:
1. Clean up per-peer compact-block in-flight state on disconnect.
FinalizeNode previously iterated mapPartialBlocks but did nothing
(a TODO comment acknowledged the gap). Peer-keyed cleanup is
actually available via listCompactBlocksInFlight, which stores
{NodeId, block hash} markers. Walk it, drop entries for the
disconnecting peer, and drop the corresponding mapPartialBlocks /
mapPartialBlocksTime entries only when no other peer is still
working on the same block. Prevents a slow memory leak when peers
churn mid-reconstruction.
2. Unify the compact block depth constant as MAX_CMPCTBLOCK_DEPTH,
shared by the cmpctblock reception check and the new low-bandwidth
MSG_CMPCT_BLOCK getdata response. Value is 100, sized to roughly
Bitcoin Core's ~50-minute wall-clock window at fluxd's 30-second
PON block spacing (Bitcoin uses 5 at 10-minute blocks). The prior
reception value of 10 covered only ~5 minutes on PON, which is too
narrow for typical mempool turnover.1 parent a1ed4cd commit 279c37f
9 files changed
Lines changed: 149 additions & 58 deletions
File tree
- src
- fluxnode
- pon
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
479 | 479 | | |
480 | 480 | | |
481 | 481 | | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
482 | 485 | | |
483 | 486 | | |
484 | 487 | | |
| |||
507 | 510 | | |
508 | 511 | | |
509 | 512 | | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
510 | 516 | | |
511 | 517 | | |
512 | 518 | | |
| |||
1809 | 1815 | | |
1810 | 1816 | | |
1811 | 1817 | | |
| 1818 | + | |
| 1819 | + | |
1812 | 1820 | | |
1813 | 1821 | | |
1814 | 1822 | | |
| |||
1823 | 1831 | | |
1824 | 1832 | | |
1825 | 1833 | | |
1826 | | - | |
| 1834 | + | |
1827 | 1835 | | |
1828 | 1836 | | |
1829 | | - | |
| 1837 | + | |
1830 | 1838 | | |
1831 | 1839 | | |
1832 | 1840 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1224 | 1224 | | |
1225 | 1225 | | |
1226 | 1226 | | |
1227 | | - | |
1228 | | - | |
| 1227 | + | |
| 1228 | + | |
| 1229 | + | |
| 1230 | + | |
1229 | 1231 | | |
1230 | 1232 | | |
1231 | 1233 | | |
| |||
1474 | 1476 | | |
1475 | 1477 | | |
1476 | 1478 | | |
1477 | | - | |
1478 | | - | |
1479 | | - | |
1480 | | - | |
1481 | | - | |
1482 | | - | |
1483 | | - | |
1484 | | - | |
1485 | 1479 | | |
1486 | | - | |
1487 | | - | |
1488 | | - | |
1489 | | - | |
1490 | | - | |
1491 | | - | |
| 1480 | + | |
| 1481 | + | |
1492 | 1482 | | |
| 1483 | + | |
| 1484 | + | |
| 1485 | + | |
| 1486 | + | |
| 1487 | + | |
| 1488 | + | |
| 1489 | + | |
| 1490 | + | |
| 1491 | + | |
| 1492 | + | |
| 1493 | + | |
1493 | 1494 | | |
1494 | 1495 | | |
| 1496 | + | |
1495 | 1497 | | |
1496 | 1498 | | |
1497 | 1499 | | |
| |||
2069 | 2071 | | |
2070 | 2072 | | |
2071 | 2073 | | |
| 2074 | + | |
| 2075 | + | |
| 2076 | + | |
| 2077 | + | |
2072 | 2078 | | |
2073 | 2079 | | |
2074 | 2080 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
634 | 634 | | |
635 | 635 | | |
636 | 636 | | |
637 | | - | |
638 | | - | |
639 | | - | |
640 | | - | |
641 | | - | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
642 | 666 | | |
643 | 667 | | |
644 | 668 | | |
| |||
6876 | 6900 | | |
6877 | 6901 | | |
6878 | 6902 | | |
| 6903 | + | |
6879 | 6904 | | |
6880 | 6905 | | |
6881 | 6906 | | |
| |||
6902 | 6927 | | |
6903 | 6928 | | |
6904 | 6929 | | |
6905 | | - | |
| 6930 | + | |
6906 | 6931 | | |
6907 | 6932 | | |
6908 | 6933 | | |
| |||
6934 | 6959 | | |
6935 | 6960 | | |
6936 | 6961 | | |
| 6962 | + | |
| 6963 | + | |
| 6964 | + | |
| 6965 | + | |
| 6966 | + | |
| 6967 | + | |
| 6968 | + | |
| 6969 | + | |
| 6970 | + | |
| 6971 | + | |
| 6972 | + | |
| 6973 | + | |
6937 | 6974 | | |
6938 | 6975 | | |
6939 | 6976 | | |
| |||
7981 | 8018 | | |
7982 | 8019 | | |
7983 | 8020 | | |
7984 | | - | |
| 8021 | + | |
7985 | 8022 | | |
7986 | 8023 | | |
7987 | 8024 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
125 | 125 | | |
126 | 126 | | |
127 | 127 | | |
128 | | - | |
129 | | - | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
130 | 137 | | |
131 | 138 | | |
132 | 139 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
137 | 137 | | |
138 | 138 | | |
139 | 139 | | |
140 | | - | |
| 140 | + | |
141 | 141 | | |
142 | 142 | | |
143 | 143 | | |
| |||
187 | 187 | | |
188 | 188 | | |
189 | 189 | | |
190 | | - | |
| 190 | + | |
191 | 191 | | |
192 | 192 | | |
193 | 193 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
28 | | - | |
| 28 | + | |
| 29 | + | |
29 | 30 | | |
30 | 31 | | |
31 | 32 | | |
| |||
139 | 140 | | |
140 | 141 | | |
141 | 142 | | |
142 | | - | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
143 | 146 | | |
144 | 147 | | |
145 | 148 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
160 | 160 | | |
161 | 161 | | |
162 | 162 | | |
163 | | - | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
164 | 170 | | |
165 | 171 | | |
166 | 172 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
49 | 56 | | |
50 | 57 | | |
51 | 58 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
892 | 892 | | |
893 | 893 | | |
894 | 894 | | |
895 | | - | |
| 895 | + | |
| 896 | + | |
| 897 | + | |
| 898 | + | |
| 899 | + | |
896 | 900 | | |
897 | | - | |
898 | 901 | | |
899 | | - | |
900 | | - | |
901 | | - | |
902 | | - | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
903 | 909 | | |
904 | 910 | | |
905 | | - | |
906 | | - | |
907 | | - | |
908 | | - | |
909 | | - | |
910 | | - | |
911 | | - | |
912 | | - | |
913 | | - | |
914 | | - | |
915 | | - | |
916 | | - | |
917 | | - | |
918 | | - | |
919 | | - | |
920 | | - | |
921 | | - | |
922 | | - | |
| 911 | + | |
| 912 | + | |
| 913 | + | |
| 914 | + | |
| 915 | + | |
| 916 | + | |
| 917 | + | |
| 918 | + | |
| 919 | + | |
| 920 | + | |
| 921 | + | |
| 922 | + | |
| 923 | + | |
| 924 | + | |
| 925 | + | |
| 926 | + | |
| 927 | + | |
| 928 | + | |
| 929 | + | |
| 930 | + | |
| 931 | + | |
| 932 | + | |
| 933 | + | |
| 934 | + | |
| 935 | + | |
| 936 | + | |
| 937 | + | |
923 | 938 | | |
924 | | - | |
925 | | - | |
| 939 | + | |
| 940 | + | |
| 941 | + | |
| 942 | + | |
926 | 943 | | |
927 | 944 | | |
928 | 945 | | |
| |||
0 commit comments