Skip to content

question about updating to mdbx 0.39.10 (0.13.8) #17419

@JkLondon

Description

@JkLondon

libmdbx released new version (0.13.8) some time ago
but there are some issues with growing inmem db size.
PR with this db #17284
Also have done some measurements of InMem DB:
Average size: 0.48 MB mdbx-go 0.39.10 (libmdbx 0.13.8)
Average size: 0.25 MB mdbx-go 0.39.9 (libmdbx 0.13.7)

Also in closed PR I've introduced testState memfix (4GB db when test is running -> 4MB)

Moreover
SaveNoSync now causing some issues


./build/bin/mdbx_chk -vvv ~/data/chiado32_full/nodes/eth69
mdbx_chk v0.13.7-0-g566b0f93 (2025-07-30T11:44:04+03:00, T-7777cbdf5aa4c1ce85ff902a4c3e6170edd42495)
Running for /.../chiado32_full/nodes/eth69 in 'read-only' mode with verbosity level 3 (of 0..9)...
 ~ meta[2] used-bytes (1228800) beyond filesize (1097728), skip it
 ~ meta[2] used-bytes (1228800) beyond filesize (1097728), skip it
 ~ filesize mismatch (expect 16777216b/1024p, have 1097728b/67p)
 ! filesize should be rounded-up to system allocation granularity 65536
 ! error -30419 (MDBX_WANNA_RECOVERY: Database should be recovered, but this could NOT be done automatically for now since it opened in read-only mode)
 ! mdbx_env_open() failed, error -30419, MDBX_WANNA_RECOVERY: Database should be recovered, but this could NOT be done automatically for now since it opened in read-only mode
Please run ./build/bin/mdbx_chk in the read-write mode (with '-w' option).

That was fixed in 0.13.8
erthink/libmdbx@43cb913

bench of mdbx-go (old - 0.13.7 new 0.13.8)

➜  mdbx git:(v0.39.9) ✗ benchstat bench-old.txt bench-new.txt 
goos: darwin
goarch: arm64
pkg: github.com/erigontech/mdbx-go/mdbx
cpu: Apple M3 Max
                         │ bench-old.txt │            bench-new.txt            │
                         │    sec/op     │   sec/op     vs base                │
Cursor-16                   106.4n ±  2%   112.3n ± 5%   +5.59% (p=0.001 n=10)
Cursor_Renew/1-16           32.72n ±  8%   32.72n ± 0%        ~ (p=0.985 n=10)
Cursor_Renew/2-16           32.29n ±  1%   32.53n ± 0%   +0.74% (p=0.022 n=10)
Cursor_Renew/3-16           115.8n ±  1%   110.2n ± 3%   -4.79% (p=0.000 n=10)
Cursor_Renew/4-16           40.23n ±  2%   39.37n ± 0%   -2.14% (p=0.000 n=10)
Cursor_Set_OneKey-16        63.18n ±  1%   62.80n ± 0%   -0.59% (p=0.002 n=10)
Cursor_Set_Sequence-16      112.5n ±  1%   110.1n ± 0%   -2.05% (p=0.000 n=10)
Cursor_Set_Random-16        555.1n ± 13%   483.0n ± 3%  -12.97% (p=0.000 n=10)
Errno_Error-16              209.0n ±  2%   192.7n ± 0%   -7.80% (p=0.000 n=10)
Txn_abort-16                172.8n ±  1%   172.3n ± 1%        ~ (p=0.404 n=10)
Txn_commit-16               50.31µ ±  6%   47.54µ ± 4%   -5.51% (p=0.011 n=10)
Txn_ro-16                   205.5n ±  3%   198.6n ± 4%   -3.33% (p=0.037 n=10)
Txn_unmanaged_abort-16      168.7n ±  2%   164.1n ± 2%   -2.67% (p=0.006 n=10)
Txn_unmanaged_commit-16     170.2n ±  1%   163.9n ± 4%   -3.70% (p=0.001 n=10)
Txn_unmanaged_ro-16         153.8n ±  2%   149.8n ± 1%   -2.63% (p=0.000 n=10)
Txn_renew-16                85.92n ±  1%   85.41n ± 0%   -0.59% (p=0.019 n=10)
Txn_Put_append-16           183.6n ±  1%   181.5n ± 0%   -1.14% (p=0.002 n=10)
Txn_Put_append_noflag-16    217.6n ±  1%   212.8n ± 1%   -2.23% (p=0.001 n=10)
Txn_Get_OneKey-16           62.17n ±  3%   58.64n ± 1%   -5.68% (p=0.000 n=10)
Txn_Get_Sequence-16         148.0n ±  2%   147.3n ± 1%   -0.47% (p=0.013 n=10)
Txn_Get_Random-16           480.5n ±  9%   463.9n ± 2%   -3.45% (p=0.023 n=10)
geomean                     167.7n         163.1n        -2.72%

                         │ bench-old.txt │            bench-new.txt            │
                         │     B/op      │    B/op     vs base                 │
Cursor-16                   16.00 ± 0%     16.00 ± 0%       ~ (p=1.000 n=10) ¹
Cursor_Renew/1-16           0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Cursor_Renew/2-16           0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Cursor_Renew/3-16           16.00 ± 0%     16.00 ± 0%       ~ (p=1.000 n=10) ¹
Cursor_Renew/4-16           0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Cursor_Set_OneKey-16        0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Cursor_Set_Sequence-16      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Cursor_Set_Random-16        0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Errno_Error-16              320.0 ± 0%     320.0 ± 0%       ~ (p=1.000 n=10) ¹
Txn_abort-16                80.00 ± 0%     80.00 ± 0%       ~ (p=1.000 n=10) ¹
Txn_commit-16               248.0 ± 0%     248.0 ± 0%       ~ (p=1.000 n=10) ¹
Txn_ro-16                   240.0 ± 0%     240.0 ± 0%       ~ (p=1.000 n=10) ¹
Txn_unmanaged_abort-16      80.00 ± 0%     80.00 ± 0%       ~ (p=1.000 n=10) ¹
Txn_unmanaged_commit-16     80.00 ± 0%     80.00 ± 0%       ~ (p=1.000 n=10) ¹
Txn_unmanaged_ro-16         80.00 ± 0%     80.00 ± 0%       ~ (p=1.000 n=10) ¹
Txn_renew-16                0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_Put_append-16           8.000 ± 0%     8.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_Put_append_noflag-16    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_Get_OneKey-16           0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_Get_Sequence-16         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_Get_Random-16           0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
geomean                                ²               +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                         │ bench-old.txt │            bench-new.txt            │
                         │   allocs/op   │ allocs/op   vs base                 │
Cursor-16                   1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=10) ¹
Cursor_Renew/1-16           0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Cursor_Renew/2-16           0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Cursor_Renew/3-16           1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=10) ¹
Cursor_Renew/4-16           0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Cursor_Set_OneKey-16        0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Cursor_Set_Sequence-16      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Cursor_Set_Random-16        0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Errno_Error-16              6.000 ± 0%     6.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_abort-16                1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_commit-16               3.000 ± 0%     3.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_ro-16                   2.000 ± 0%     2.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_unmanaged_abort-16      1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_unmanaged_commit-16     1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_unmanaged_ro-16         1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_renew-16                0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_Put_append-16           1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_Put_append_noflag-16    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_Get_OneKey-16           0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_Get_Sequence-16         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Txn_Get_Random-16           0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
geomean                                ²               +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

TODO:

  • bench with @sudeepdino008 tool
  • check sizes of synced node in diff regimes

FYI @AskAlexSharov

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions