perf: reduce murmur call overhead and clarify checksum usage - #24
Merged
Conversation
Murmur's public update and digest methods stayed out of line in the measured consumer build, preventing short writes from optimizing with their callers. Give these six hot methods ordinary inline hints, as the other streaming families already do. With rustc 1.99.0-nightly (3d6c19bb9) on aarch64-apple-darwin, no LTO, and 16 codegen units, three alternating runs of the existing streaming benchmarks reduced short-chunk median latency by 6-42%. For example, x86_32 at 241 B / 17 B chunks went from 123.4 ns to 71.36 ns, while x64_128 at 4 KiB / 64 B chunks went from 639.0 ns to 521.5 ns. Bulk-input results stayed within measurement noise; the repository's ThinLTO configuration also showed no regression in the comparison run. Verified with cargo x test, including no_std and optimized reference comparisons. The changelog describes the measured consumer benefit.
The unit and integration suites repeated the same three RFC input/digest pairs. Keep one authoritative vector table in the library tests and move the integration-only binary vector there so it also runs under no_std and Miri. Retain the integration matrix against the independent fnv implementation and all streaming, offset-basis, and reset coverage.
Stable integer digests do not define a wire format: xxHash uses canonical big-endian bytes, FNV specifies little-endian bytes, and Murmur's output word order differs from formatting its returned integer. Explain the required conversions in the family docs and the portability guide. Rust Hash also adds type framing and can change its encoding across compiler versions, even on the same target. Direct checksum callers to the raw-byte APIs instead of assuming hash_one computes the same input. Show how to format MD5's byte array as the conventional 32-character checksum, including leading zeroes, using existing Rust APIs. Exercise the example as a doctest without introducing a digest wrapper or a new dependency.
The architecture guide still listed four families and claimed every 128-bit result was u128, omitting MD5's byte digest and finalization. It also claimed unsafe code was confined to SIMD intrinsics even though XXH32's block read and XXH3's validated secret reads are outside the kernel layer. Describe the actual boundaries so maintainers do not rely on those inaccurate assumptions. Include opt-in family/std features and the shared I/O adapter location, and account for MD5's independent reference and RFC-vector coverage in the README.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reduce short-chunk MurmurHash3 overhead by allowing its public
updateanddigestmethods to inline across crates. Three alternating benchmark runs without LTO on AArch64 showed up to 42% lower latency in the existing short-chunk cases; the performance commit records the toolchain and representative results.Clarify portable checksum inputs, digest byte order, and MD5 hexadecimal formatting. Consolidate duplicated FNV specification vectors while retaining their unique cases and independent reference comparisons, and correct stale architecture descriptions.
The changes are split into four small commits, each explaining its motivation.