-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Closed
Description
libmdbx released new version (0.13.8) some time ago
but there are some issues with growing inmem db size.
PR with this db #17284
Also have done some measurements of InMem DB:
Average size: 0.48 MB mdbx-go 0.39.10 (libmdbx 0.13.8)
Average size: 0.25 MB mdbx-go 0.39.9 (libmdbx 0.13.7)
Also in closed PR I've introduced testState memfix (4GB db when test is running -> 4MB)
Moreover
SaveNoSync now causing some issues
./build/bin/mdbx_chk -vvv ~/data/chiado32_full/nodes/eth69
mdbx_chk v0.13.7-0-g566b0f93 (2025-07-30T11:44:04+03:00, T-7777cbdf5aa4c1ce85ff902a4c3e6170edd42495)
Running for /.../chiado32_full/nodes/eth69 in 'read-only' mode with verbosity level 3 (of 0..9)...
~ meta[2] used-bytes (1228800) beyond filesize (1097728), skip it
~ meta[2] used-bytes (1228800) beyond filesize (1097728), skip it
~ filesize mismatch (expect 16777216b/1024p, have 1097728b/67p)
! filesize should be rounded-up to system allocation granularity 65536
! error -30419 (MDBX_WANNA_RECOVERY: Database should be recovered, but this could NOT be done automatically for now since it opened in read-only mode)
! mdbx_env_open() failed, error -30419, MDBX_WANNA_RECOVERY: Database should be recovered, but this could NOT be done automatically for now since it opened in read-only mode
Please run ./build/bin/mdbx_chk in the read-write mode (with '-w' option).
That was fixed in 0.13.8
erthink/libmdbx@43cb913
bench of mdbx-go (old - 0.13.7 new 0.13.8)
➜ mdbx git:(v0.39.9) ✗ benchstat bench-old.txt bench-new.txt
goos: darwin
goarch: arm64
pkg: github.com/erigontech/mdbx-go/mdbx
cpu: Apple M3 Max
│ bench-old.txt │ bench-new.txt │
│ sec/op │ sec/op vs base │
Cursor-16 106.4n ± 2% 112.3n ± 5% +5.59% (p=0.001 n=10)
Cursor_Renew/1-16 32.72n ± 8% 32.72n ± 0% ~ (p=0.985 n=10)
Cursor_Renew/2-16 32.29n ± 1% 32.53n ± 0% +0.74% (p=0.022 n=10)
Cursor_Renew/3-16 115.8n ± 1% 110.2n ± 3% -4.79% (p=0.000 n=10)
Cursor_Renew/4-16 40.23n ± 2% 39.37n ± 0% -2.14% (p=0.000 n=10)
Cursor_Set_OneKey-16 63.18n ± 1% 62.80n ± 0% -0.59% (p=0.002 n=10)
Cursor_Set_Sequence-16 112.5n ± 1% 110.1n ± 0% -2.05% (p=0.000 n=10)
Cursor_Set_Random-16 555.1n ± 13% 483.0n ± 3% -12.97% (p=0.000 n=10)
Errno_Error-16 209.0n ± 2% 192.7n ± 0% -7.80% (p=0.000 n=10)
Txn_abort-16 172.8n ± 1% 172.3n ± 1% ~ (p=0.404 n=10)
Txn_commit-16 50.31µ ± 6% 47.54µ ± 4% -5.51% (p=0.011 n=10)
Txn_ro-16 205.5n ± 3% 198.6n ± 4% -3.33% (p=0.037 n=10)
Txn_unmanaged_abort-16 168.7n ± 2% 164.1n ± 2% -2.67% (p=0.006 n=10)
Txn_unmanaged_commit-16 170.2n ± 1% 163.9n ± 4% -3.70% (p=0.001 n=10)
Txn_unmanaged_ro-16 153.8n ± 2% 149.8n ± 1% -2.63% (p=0.000 n=10)
Txn_renew-16 85.92n ± 1% 85.41n ± 0% -0.59% (p=0.019 n=10)
Txn_Put_append-16 183.6n ± 1% 181.5n ± 0% -1.14% (p=0.002 n=10)
Txn_Put_append_noflag-16 217.6n ± 1% 212.8n ± 1% -2.23% (p=0.001 n=10)
Txn_Get_OneKey-16 62.17n ± 3% 58.64n ± 1% -5.68% (p=0.000 n=10)
Txn_Get_Sequence-16 148.0n ± 2% 147.3n ± 1% -0.47% (p=0.013 n=10)
Txn_Get_Random-16 480.5n ± 9% 463.9n ± 2% -3.45% (p=0.023 n=10)
geomean 167.7n 163.1n -2.72%
│ bench-old.txt │ bench-new.txt │
│ B/op │ B/op vs base │
Cursor-16 16.00 ± 0% 16.00 ± 0% ~ (p=1.000 n=10) ¹
Cursor_Renew/1-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Cursor_Renew/2-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Cursor_Renew/3-16 16.00 ± 0% 16.00 ± 0% ~ (p=1.000 n=10) ¹
Cursor_Renew/4-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Cursor_Set_OneKey-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Cursor_Set_Sequence-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Cursor_Set_Random-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Errno_Error-16 320.0 ± 0% 320.0 ± 0% ~ (p=1.000 n=10) ¹
Txn_abort-16 80.00 ± 0% 80.00 ± 0% ~ (p=1.000 n=10) ¹
Txn_commit-16 248.0 ± 0% 248.0 ± 0% ~ (p=1.000 n=10) ¹
Txn_ro-16 240.0 ± 0% 240.0 ± 0% ~ (p=1.000 n=10) ¹
Txn_unmanaged_abort-16 80.00 ± 0% 80.00 ± 0% ~ (p=1.000 n=10) ¹
Txn_unmanaged_commit-16 80.00 ± 0% 80.00 ± 0% ~ (p=1.000 n=10) ¹
Txn_unmanaged_ro-16 80.00 ± 0% 80.00 ± 0% ~ (p=1.000 n=10) ¹
Txn_renew-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_Put_append-16 8.000 ± 0% 8.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_Put_append_noflag-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_Get_OneKey-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_Get_Sequence-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_Get_Random-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
geomean ² +0.00% ²
¹ all samples are equal
² summaries must be >0 to compute geomean
│ bench-old.txt │ bench-new.txt │
│ allocs/op │ allocs/op vs base │
Cursor-16 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=10) ¹
Cursor_Renew/1-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Cursor_Renew/2-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Cursor_Renew/3-16 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=10) ¹
Cursor_Renew/4-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Cursor_Set_OneKey-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Cursor_Set_Sequence-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Cursor_Set_Random-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Errno_Error-16 6.000 ± 0% 6.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_abort-16 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_commit-16 3.000 ± 0% 3.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_ro-16 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_unmanaged_abort-16 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_unmanaged_commit-16 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_unmanaged_ro-16 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_renew-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_Put_append-16 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_Put_append_noflag-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_Get_OneKey-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_Get_Sequence-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
Txn_Get_Random-16 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
geomean ² +0.00% ²
¹ all samples are equal
² summaries must be >0 to compute geomean
TODO:
- bench with @sudeepdino008 tool
- check sizes of synced node in diff regimes
FYI @AskAlexSharov
Metadata
Metadata
Assignees
Labels
No labels