Describe the bug
testMergeTwoSegsWithASingleDocPerSeg fails deterministically with seed 599D21DD0964F401
Steps/Code to reproduce bug
cd java/cuvs-lucene
mvn test -Dtest=TestLucene99AcceleratedHNSWVectorsFormat#testMergeTwoSegsWithASingleDocPerSeg -Dtests.seed=599D21DD0964F401
Failure:
arrays first differed at element [0]; expected:<0.9042659> but was:<0.8326107>
Expected behavior
Test passes.
Environment details (please complete the following information):
- originally found on arm64 PR build, but reproduced on amd64.
Additional context
Confirmed via a local diagnostic patch that 0.9042659 is f[0][0] (doc 0's indexed vector) and 0.8326107 is exactly f[1][0] (doc 1's indexed vector) — values.vectorValue(0) after merge returns doc 1's vector verbatim. This looks like a doc-ordinal swap.
getFloatVectorValues/vectorValue() are served entirely by the standard Lucene flat-vector format (not CuVS-specific graph code), so the bug is in how the flat storage gets merged. Lucene99AcceleratedHNSWVectorsWriter.mergeOneField() runs two independent merge iterations over the same MergeState/field — flatVectorsWriter.mergeOneField(fieldInfo, mergeState), then vectorBasedMerge(fieldInfo, mergeState) (which separately calls KnnVectorsWriter.MergedVectorValues.mergeFloatVectorValues in Utils.createListFromMergedVectors). If the underlying flat writer defers its actual merge to finish(), the second (CAGRA-build) iteration could run first and disturb shared per-segment reader/iterator state before the flat writer's own (later) pass executes - worth checking whether mergeState's vector readers are safely re-iterable across two independent merge passes for the same field.
There is a similar error in TestLucene99AcceleratedHNSWVectorsFormat.testTwoVectorFieldsPerDoc with seed EEBDAA2298537BF7.
Describe the bug
testMergeTwoSegsWithASingleDocPerSegfails deterministically with seed599D21DD0964F401Steps/Code to reproduce bug
Failure:
Expected behavior
Test passes.
Environment details (please complete the following information):
Additional context
Confirmed via a local diagnostic patch that
0.9042659isf[0][0](doc 0's indexed vector) and0.8326107is exactlyf[1][0](doc 1's indexed vector) — values.vectorValue(0) after merge returns doc 1's vector verbatim. This looks like a doc-ordinal swap.getFloatVectorValues/vectorValue()are served entirely by the standard Lucene flat-vector format (not CuVS-specific graph code), so the bug is in how the flat storage gets merged.Lucene99AcceleratedHNSWVectorsWriter.mergeOneField()runs two independent merge iterations over the same MergeState/field —flatVectorsWriter.mergeOneField(fieldInfo, mergeState), thenvectorBasedMerge(fieldInfo, mergeState)(which separately callsKnnVectorsWriter.MergedVectorValues.mergeFloatVectorValuesinUtils.createListFromMergedVectors). If the underlying flat writer defers its actual merge tofinish(), the second (CAGRA-build) iteration could run first and disturb shared per-segment reader/iterator state before the flat writer's own (later) pass executes - worth checking whether mergeState's vector readers are safely re-iterable across two independent merge passes for the same field.There is a similar error in
TestLucene99AcceleratedHNSWVectorsFormat.testTwoVectorFieldsPerDocwith seedEEBDAA2298537BF7.