Repository navigation
Conversation
The big-endian branches were never compiled and no longer build: direct_bitset.rs uses copy_nonoverlapping without importing it and gates lvl_get_item to little-endian while its callers are unconditional, bitset/serialization.rs calls a non-existent chunked_read with a non-existent data_blocks_len. The remaining branches are unimplemented!(). Implement them so that the serialized format is the same on both byte orders (every u64 lane stored little-endian, lanes in array order): - BitBlock::to_le_bytes/from_le_bytes for the wide SIMD blocks convert lane-wise. BitBlock::to_le now reinterprets the little-endian bytes natively (from_ne_bytes) instead of round-tripping back to the original value; BufferedWriter relies on that. - Writer::write_primitives/write_masks and Reader::read_primitives/read_masks/foreach_read_mask convert per element on big-endian. foreach_read_mask needs T: BitBlock for that. - BitSet::deserialize reads the data blocks through foreach_read_mask on big-endian. - DirectBitset::read_mask reads the byte array and converts it. The old ALIGNED branch also passed size_of::<Mask>() as a count of masks, not bytes. lvl_get_item needs no index correction: sub-masks are stored as little-endian u64 in array order and read_mask::<u64> already converts them. Tested on FreeBSD/powerpc64 (big-endian, POWER9): the library tests pass, and data serialized on x86-64 deserializes identically through BitSet, ImmutableBitset and DirectBitset (aligned and unaligned), and vice versa.
Owner
|
Do you REALLY need big-endian? How do you test it? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The big-endian branches were never compiled and no longer build:
direct_bitset.rs uses copy_nonoverlapping without importing it and
gates lvl_get_item to little-endian while its callers are unconditional,
bitset/serialization.rs calls a non-existent chunked_read with a
non-existent data_blocks_len. The remaining branches are
unimplemented!().
Implement them so that the serialized format is the same on both byte
orders (every u64 lane stored little-endian, lanes in array order):
lane-wise. BitBlock::to_le now reinterprets the little-endian bytes
natively (from_ne_bytes) instead of round-tripping back to the
original value; BufferedWriter relies on that.
Reader::read_primitives/read_masks/foreach_read_mask convert per
element on big-endian. foreach_read_mask needs T: BitBlock for that.
big-endian.
ALIGNED branch also passed size_of::() as a count of masks, not
bytes. lvl_get_item needs no index correction: sub-masks are stored as
little-endian u64 in array order and read_mask:: already converts
them.
Tested on FreeBSD/powerpc64 (big-endian, POWER9): the library tests
pass, and data serialized on x86-64 deserializes identically through
BitSet, ImmutableBitset and DirectBitset (aligned and unaligned), and
vice versa.