Skip to content

refactor(xor): remove persisted item counts - #795

Open
tisonkun wants to merge 2 commits into
apache:mainfrom
tisonkun:codex/xor-remove-item-count
Open

tisonkun wants to merge 2 commits into
apache:mainfrom
tisonkun:codex/xor-remove-item-count

Conversation

@tisonkun

@tisonkun tisonkun commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

XOR membership queries need the construction seed, segment layout, and fingerprints, but not the number of input hashes. Requiring a count in the serialized image prevents importing those fields without inventing metadata. xorf and FastFilter C also omit the count. This is a representation choice: FastFilter Java retains its input size to derive the layout.

Remove the finished filter's count and getNumItems(), isEmpty(), and getBitsPerItem(). Callers measuring bits per input can divide the footprint by their known input count. The builder still tracks its updates.

XOR has not been released, so serialization stays at version 1. Reduce the header from 24 to 16 bytes, storing the fingerprint width, segment length, and seed. The Rust counterpart uses the same layout for 8/16-bit filters.

Validation: mvn -B clean verify passes all 2,335 tests, Javadocs, and license checks. XOR tests cover heapify/wrap round trips, reference fingerprints, malformed headers, and truncated images.

@tisonkun

tisonkun commented Nov 5, 2026 •

Copy link
Copy Markdown
Member Author

The Rust counterpart uses the same header and serialization version 1. Its serialization tests consume scalar and UTF-8/raw-byte fixtures generated from this PR's c179c5bc0 and verify byte-for-byte construction and round trips.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant