Modules: fixed double-free in shared dict update with eviction.#1041
Merged
xeioex merged 3 commits intonginx:masterfrom Mar 30, 2026
Merged
Modules: fixed double-free in shared dict update with eviction.#1041xeioex merged 3 commits intonginx:masterfrom
xeioex merged 3 commits intonginx:masterfrom
Conversation
Previously, when updating an existing key's string value in a shared dictionary with timeout and evict enabled, ngx_js_dict_alloc() could trigger ngx_js_dict_evict() if the zone was full. Since the node being updated was still in the expire tree, eviction could free it. The subsequent ngx_slab_free_locked() call in the update path then freed the already-freed string data, causing the "chunk is already free" alert followed by a segfault. The fix removes the node from the expire tree before allocating memory for the new value, preventing eviction from reaching it. On allocation failure the node is re-inserted with its original expiry time.
Previously, when a slab allocation failed in evict mode, only 16
entries were evicted with a single retry. This could still result
in SharedMemoryError when the freed slab slots did not match the
requested allocation size class, even though the zone had plenty
of evictable entries.
In practice, it might happen when the following conditions are met:
- The shared zone is full
- evict flag is enabled
- key/value entries differ in size
The allocation now retries in a loop, evicting 16 entries at a time,
until the allocation succeeds or no more entries remain in the expire
tree.
After this change, allocation with evict enabled can only fail when:
- the value is larger than the zone's usable space
- the expire tree has no entries left to evict
- zone metadata overhead leaves insufficient room
Previously, keys(), items(), and size() called ngx_js_dict_expire() under a read lock. Since ngx_js_dict_expire() deletes nodes from both rbtrees and frees slab memory, concurrent readers on different worker processes could corrupt shared memory by freeing the same expired nodes simultaneously. The fix removes ngx_js_dict_expire() calls from all read-locked paths and instead skips expired entries during iteration, consistent with how get() and has() already handle expiry. Actual cleanup of expired entries is deferred to write-side operations (set, add, delete, clear).
8f4f0e3 to
ccf787e
Compare
Contributor
|
May be it would be nice add extra regression test for the third change, covering keys()/items()/size() with expired-but-not-yet-reclaimed entries, ideally for both njs and qjs paths. Otherwise looks good. |
Contributor
Author
Yes, I considered it and decided not to add more tests as existing tests already cover what we need the most. expired-but-not-yet-reclaimed - is hard to observe the internal state of whether nodes were reclaimed or just skipped. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When updating an existing key's string value in a shared dictionary with timeout and evict enabled, ngx_js_dict_alloc() could trigger ngx_js_dict_evict() if the zone was full. Since the node being updated was still in the expire tree, eviction could free it. The subsequent ngx_slab_free_locked() call in the update path then freed the already-freed string data, causing the "chunk is already free" alert followed by a segfault.
The issue was more likely to occur when the zone fills to capacity and eviction is triggered during the update path.
This closes #1036 issue on Github.