Refactor dict restoring abstraction#3561
Conversation
| * the "htdict" prefix is used to avoid colliding with the "dict" in libvalkey */ | ||
| #define dictCreate(type) htdictCreate(type) | ||
| #define dictExpand(d, size) htdictExpand(d, size) | ||
| #define dictSetKey(d, de, key) htdictSetKey(d, de, key) |
There was a problem hiding this comment.
I want to eliminate this function. "set key" is an anti-pattern for a dict structure. The only place this is being used is for one use-case in expire.c. We could have fixed that with the key dup callback, but that callback was eliminated in the transition to hashtable.
Might just need a better API which addresses the use case in expire.
There was a problem hiding this comment.
Or would it be feasible to refactor that use case to use hashtable directly?
I got slightly distracted - turns out it was easy. I had a PR ready (#3566) before I remembered I was reviewing here 😓
There was a problem hiding this comment.
@rainsupreme I see you solved it by creating a dict clone based on hashtable where the dictEntry clone is not opaque to your code.
I'd call that's a workaround for something you can't do in dict anymore.
IMHO we could just keep dictEntry non-opaque and then it's strait-forward to just replace the individual calls to functions like dictAddOrFind with the lower-level hashtable functions + manual dictEntry manipulation whenever the dict API is limiting you.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## unstable #3561 +/- ##
============================================
+ Coverage 76.35% 76.63% +0.27%
============================================
Files 159 159
Lines 80054 80135 +81
============================================
+ Hits 61125 61408 +283
+ Misses 18929 18727 -202
🚀 New features to boost your workflow:
|
Signed-off-by: Jim Brunner <brunnerj@amazon.com>
There was a problem hiding this comment.
Looks good overall, though I think there are couple things to fix. I do think keeping the opaqueness of dict is the right move, and we don't have to give up the performance gain either!
I'm a little disappointed with the dict naming conflict from libvalkey, and all the #define boilerplate in dict.h. We didn't have to do this before this change (apparently?), so my (unresearched) hunch is that there should be some way to avoid the boilerplate. Could we remove dict.h from server.h and only include it where dict is used, or something like that? 🤔
| * the "htdict" prefix is used to avoid colliding with the "dict" in libvalkey */ | ||
| #define dictCreate(type) htdictCreate(type) | ||
| #define dictExpand(d, size) htdictExpand(d, size) | ||
| #define dictSetKey(d, de, key) htdictSetKey(d, de, key) |
There was a problem hiding this comment.
Or would it be feasible to refactor that use case to use hashtable directly?
I got slightly distracted - turns out it was easy. I had a PR ready (#3566) before I remembered I was reviewing here 😓
Signed-off-by: Jim Brunner <brunnerj@amazon.com>
Signed-off-by: Jim Brunner <brunnerj@amazon.com>
zuiderkwast
left a comment
zuiderkwast
left a comment
There was a problem hiding this comment.
Much effort for very little gain IMHO, but I won't refuse it. :)
| db.o \ | ||
| debug.o \ | ||
| defrag.o \ | ||
| dict_ht.o \ |
There was a problem hiding this comment.
Don't mix spaces and tabs.
|
|
||
|
|
||
| dict *htdictCreate(dictType *type) { | ||
| type->entryGetKey = htdictEntryGetKey; |
There was a problem hiding this comment.
This manipulates the dictType which isn't owned by this dict instance. It's it's shared among multiple dicts. It isn't marked as const but still...
Is this ugliness the cost this PR is paying for removing some other uglinesses?
I guess it can be OK, but it's worth mentioning. :)
| #define UNUSED(V) ((void)V) | ||
|
|
||
| /* Callback for dictType.entryGetKey, which expects void pointers. */ | ||
| const void *htdictEntryGetKey(const void *entry) { |
There was a problem hiding this comment.
It can be marked as static right?
There was a problem hiding this comment.
Woudn't it be nicer if the filename and the function prefix htdict would match? E.g.
htdictCreate/htdict.cdictHtCreate/dict_ht.cdicthtCreate/dictht.c.
3 participants
The recent update to
dict(#3366) improves performance by makingdicta thin wrapper on top of thehashtableimplementation.As part of that refactoring, we lost some of our
dictabstraction. ThedictEntrybecame public (again). The defrag code was diving into the entry directly.This update:
dictabstraction by makingdictEntryopaque (again)dict(out of defrag).cfile rather than the.hfile (allowing for opaqueness in the data structure and code)dictEntryGetKeyon everydict(it's essentially a required constant)Link time optimization (LTO) will result in the same inlining of functions, however it can now use configurable options to tune the level of inlining. This potentially reduces L1 cache bloat.