Repository navigation
Support 32-bit machines - #128
Merged
Merged
Conversation
The crate refused to compile unless `target_pointer_width` was 64, via a `compile_error!` in `lib.rs` and `#![cfg(target_pointer_width = "64")]` on most modules. Removing them exposed only a handful of real dependencies on the pointer width, all of which are fixed here. - `broadword`: the 64-bit step constants were typed `usize`, so they overflowed at compile time on a 32-bit target. They are now `u64`, along with the helpers that consume them. On a 64-bit machine the types are identical, so nothing changes there. - `Rank9SelIndex::block_rank_pairs`: it packs seven 9-bit subranks into one word, which does not fit in a 32-bit `usize`. It is now `Vec<u64>`. - `DArrayIndex::block_inventory`: bit positions stored as `isize` would wrap into the negative overflow marker beyond 2^31 bits on a 32-bit machine. It is now `Vec<i64>`. - `EliasFanoBuilder::push`: `val as usize >> low_len` cast before shifting and thus truncated the value. It now shifts before casting. - `EliasFano::delta`: it computed the delta in `usize`, which overflows when `low_len >= 32`. It now computes in `u64`. `usize` and `isize` are serialized as fixed 64-bit little-endian integers instead of native-width ones, so the format no longer depends on the machine. The bytes are unchanged from those written by 0.9.1 on a 64-bit machine, so existing data stay readable; deserializing a value that does not fit in `usize` of the machine now fails with `SucdsError::InvalidArgument`. Verified that the serialized bytes and the query results are identical between 0.9.1 on 64-bit, this commit on 64-bit, and this commit on a 32-bit target, and that all unit tests pass on wasm32-wasip1. CI gains a `test-32bit` job that builds and runs the tests for i686-unknown-linux-gnu (i686 binaries run natively on the x86_64 runner, so the tests really execute with a 32-bit `usize`), and the no_std job also builds for thumbv7em-none-eabi to cover no_std and a 32-bit pointer width at once. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Say the same in two short paragraphs instead of eight lines of prose. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Removes the 64-bit-only restriction: the
compile_error!inlib.rsand the#![cfg(target_pointer_width = \"64\")]attributes on 14 modules are gone, and the handful of real dependencies on the pointer width are fixed.Why
The restriction turned out to be almost entirely incidental. Deleting the
cfggates and building for a 32-bit target produced only three compile errors, all of them broadword constants. Everything else was already stored in explicit widths (Vec<u64>,Vec<u16>,Vec<u8>), withusizeused for lengths and positions as it should be.What actually depended on the pointer width
broadword— the 64-bit step constants (ONES_STEP_8,MSBS_STEP_9, …) were typedusize, so they overflowed at compile time on a 32-bit target. Nowu64, along with the helpers that consume them. On a 64-bit machine the two types are identical, so this is a no-op there.Rank9SelIndex::block_rank_pairs— packs seven 9-bit subranks into one word, which does not fit in a 32-bitusize. NowVec<u64>.DArrayIndex::block_inventory— bit positions stored asisize; beyond 2^31 bits on a 32-bit machine they would wrap into the negative overflow marker and silently corruptselect. NowVec<i64>.EliasFanoBuilder::push—val as usize >> low_lencast before shifting, truncating the value. Now shifts before casting. (Found by an actual 32-bit test run, not by reading.)EliasFano::delta— computed the delta inusize, which overflows oncelow_len >= 32. Now computes inu64. (Also found by running.)Serialization
usizeandisizeare now serialized as fixed 64-bit little-endian integers instead of native-width ones, so the format no longer depends on the machine.usizeof the machine fails withSucdsError::InvalidArgumentrather than truncating silently.Verification
Runtime behavior was checked on
wasm32-wasip1, a genuine 32-bit target, run under Node's WASI.intrinsics/no-default-features)cargo check --target i686-unknown-linux-gnu --all-targets(stable and 1.62.0)cargo build --no-default-features --target thumbv7em-none-eabi(stable and 1.62.0)x86_64-unknown-none(existing no_std job)cargo fmt/ clippy nursery-D warnings/cargo doc -D warningsSerialized bytes and query results were compared across three builds — 0.9.1 on 64-bit, this branch on 64-bit, and this branch on 32-bit — over
BitVector,Rank9Sel,DArray,SArray,CompactVector,DacsByte,DacsOpt,PrefixSummedEliasFano,EliasFano, andWaveletMatrix. All three outputs are byte-identical.CI
test-32bitjob: builds and runs the tests fori686-unknown-linux-gnu. i686 binaries execute natively on the x86_64 runner, so the tests really run with a 32-bitusizerather than only cross-compiling.-C target-cpu=nativeis deliberately not set for theintrinsicsrun, since the host is x86_64. Stable only — the MSRV is covered by the 64-bittestjob, and nothing here depends on the language version.no-stdjob also builds forthumbv7em-none-eabi, covering no_std and a 32-bit pointer width at once.Note: the i686 job itself has not been executed yet — it was developed on macOS, where no Linux runner is available. It is compile-checked on both toolchains, and the 32-bit runtime behavior is covered by the wasm32 run above. This PR's CI is its first real execution.
Remaining limitations (documented in the README and crate docs)
usize, i.e. fewer than 2^32 stored bits or integers.usizeandu64are the same type there and the memory layout is unchanged. Benchmarks have not been run.🤖 Generated with Claude Code