Add wyhash2 and metro64/128 modern hashing #2293
Conversation
Just for info: |
Now that's what I call modern! Spent some time today and yesterday implementing a5hash and komihash, so there are even more options to choose from (sorry to the reviewers 😄). |
|
That about wraps up the initial code honing and few passes. I won't add on top of this PR any further now, given no more CI issues or review feedback pop up. Looks ready to go! |
lib/std/hash/metro128.c3
Outdated
|
|
||
| for (; data.len >= 32; data = data[32:^32]) | ||
| { | ||
| self.state[0] += ((ulong*)data.ptr)[0] * K[0]; self.state[0] = self.state[0].rotr(29) + self.state[2]; |
There was a problem hiding this comment.
This is unaligned, so you need @unaligned_load
|
|
||
| if (final_data.len >= 16) | ||
| { | ||
| self.state[0] += ((ulong*)final_data.ptr)[0] * K[2]; self.state[0] = self.state[0].rotr(33) * K[3]; |
There was a problem hiding this comment.
Not sure, but maybe this needs unaligned_load
lib/std/hash/metro128.c3
Outdated
|
|
||
| if (final_data.len >= 8) | ||
| { | ||
| self.state[0] += ((ulong*)final_data.ptr)[0] * K[2]; self.state[0] = self.state[0].rotr(33) * K[3]; |
|
I found even more examples of unaligned load, so try to look through and fix those. |
|
A good way to test this is to pass in data that is unaligned, for example: char[] data_aligned = "XThe data";
char[] data = data_aligned[1..]; // This is pretty sure to be unaligned. |
# Conflicts: # releasenotes.md
|
Please check the benchmarks now that things actually use unaligned access properly! I think I fixed all the bugs, so it's time to merge this! |
Updating the C3 hash kit with some more modern hashes that are performant, random, and work reliably according to their known test vectors.
For this work, I'd like to give a special thanks to Dr. Timofey Prodanov for this article and the related benchmarks spreadsheet. Both of these items are important, interesting, and relevant for this change.
Typical benchmark results:
@lerno Given the benchmarking results when run and Dr. Prodanov's tests, as well as the poor randomness and speed of FNV hashing, I would like to propose a change - in a different pull request - for how types are hashed by default in stdlib.
If we use wyhash2 for types up to and including 16 bytes in size and metro64 for vectors, arrays, etc. beyond that,If we reevaluate which hashes in this PR are best at particular input sizes, based on speed and randomness, C3 can have much faster and more random hashing performance (which would hopefully speed a lot of things up)!The wyhash2 speed changes are DISGUSTING - so are Metro's at a higher data size!
Anyway, just a thought. Let me know what you think.