Repository navigation
test: goldens that grow a lineage and draw a clone size - #52
Merged
Merged
Conversation
The three entries explained edits from the affinity-translation branch, which is now in master, so they explain nothing and would excuse the next edit to the same names. The gate only runs this audit on a pull request and only with --declarations, so master always looks green and the sweep has no signal telling you it is owed.
The cross-platform claim is only made about code the fixtures run, and until now no fixture grew a lineage or drew a clone size. That is precisely where the engine's transcendental arithmetic is: lineage/poisson.rs:26 exp every offspring count lineage/affinity.rs:209,221 exp every affinity lineage/clone_size.rs:33 ln, sqrt, cos lognormal sizes lineage/clone_size.rs:43,46 powf power-law sizes lineage/clone_size.rs:52 exp lognormal sizes Rust documents the precision of all of these as platform-dependent, and nothing rounds the difference away afterwards: a clone size is `x.round()` of a `powf`, so one ulp is a different integer, and `lineage_affinity` is exported at full f64 precision. A fixed SplitMix stream makes the draws identical everywhere. It says nothing about what those draws are then turned into, which is the half of the promise in determinism.rs -- "one seed and one determinism version give one answer, on every platform the library builds on" -- that had no test at all. The symptom was in plain sight: determinism versions 3 and 4 both passed through this file moving nothing but the determinism_version column. The fixtures could not see the model they were stamping. Four fixtures, kept deliberately small, chosen to reach all five sites: a neutral lineage (10 rows, varying lineage_abundance); a selected one (12 rows, lineage_affinity at sixteen significant digits, e.g. 0.8564151774836135, which is the sharpest probe available); a power-law repertoire (duplicate_count 4, 1, 1, 1); and a lognormal one (4, 1, 15, 5), which reaches Box-Muller's ln, sqrt and cos where powf alone does not. Each was checked to vary rather than sit constant -- a fixture whose column is the same value throughout would pass on every platform while testing none of this. This answers nothing by itself. CI runs ubuntu, macos and windows, so pushing it is the experiment. Green means the guarantee holds for these paths on these platforms and is checked from here on rather than asserted. Red on one of them means the headline promise is false for lineage output, and what to do about it -- pin an audited portable implementation, commit canonical tables, or narrow the documented promise -- is a decision, not a cleanup. Four fixtures on three operating systems and one architecture is not a proof. It does not cover aarch64, another libm, or inputs sitting next to a rounding threshold.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
determinism.rspromises: "one seed and one determinism version give oneanswer, on every platform the library builds on." This PR tests the second
half of that sentence for the clonal paths, where it has never been tested.
The gap
The cross-platform check —
test_output_matches_the_committed_golden, the onlything in the suite that compares platforms at all — can only cover code the
fixtures run. No fixture grew a lineage or drew a clone size. That is
exactly where the engine's transcendental arithmetic lives:
lineage/poisson.rs:26explineage/affinity.rs:209,221explineage/clone_size.rs:33ln,sqrt,coslineage/clone_size.rs:43,46powflineage/clone_size.rs:52expRust documents the precision of all of these as platform-dependent. Nothing
rounds the difference away afterwards — a clone size is
x.round()of apowf(
clone_size.rs:47), so one ulp is a different integer, not a differentlast digit, and
lineage_affinityis exported at full f64 precision.A fixed SplitMix stream makes the draws identical everywhere. It says nothing
about what those draws are then turned into.
The symptom was in plain sight: determinism versions 3 and 4 both passed
through
tests/support/golden.pymoving nothing but thedeterminism_versioncolumn. The fixtures could not see the model they were stamping.
golden.py's own docstring already names the principle — "the cross-platformclaim is only made about code the fixtures run" — and records an earlier
instance of the same gap, when no fixture drew a substitution and changing the
replacement-base model for every substitution pass moved no golden byte.
The experiment
Four fixtures, deliberately small, chosen to reach all five sites:
clonal_lineagepoisson.rsexplineage_abundancevariesclonal_lineage_selectedaffinity.rsexplineage_affinityat full f64 —0.8564151774836135clonal_power_lawclone_size.rspowfduplicate_count4, 1, 1, 1clonal_lognormalclone_size.rsln/sqrt/cos/expduplicate_count4, 1, 15, 5Each was checked to vary rather than sit constant: a fixture whose column holds
one value throughout would pass on every platform while testing none of this.
lineage_affinityis the sharpest probe available — sixteen significant digitsof
exp(-beta · distance), compared byte for byte.CI runs ubuntu, macos and windows, so merging this is not what matters —
running it is.
Reading the result
and from here on it is checked rather than asserted: the next change to the
growth or clone-size arithmetic cannot move a platform silently.
lineage output and every clonal dataset's reproducibility claim is
platform-local. What to do then — pin an audited portable implementation,
commit canonical tables, or narrow the documented promise — is a decision, and
deliberately not bundled here.
This is also why it was kept out of #50 and #51 even though both touched this
arithmetic: a finding about portability should not arrive as a red run on a
defect fix.
What this does not establish
Four fixtures, three operating systems, one architecture. It does not cover
aarch64, a different libm, a different Rust version, or the parameter space of
each law. Inputs sitting next to a rounding threshold — a clone size landing on
.5, an affinity near underflow — are untested. Finite goldens support apolicy; they do not prove it.
Also here
The three inventory declarations from #51 are swept. Their changes are in the
baseline now, so they explain nothing and would excuse the next edit to the same
names. That audit runs only on a pull request and only with
--declarations, somaster always looks green and the sweep has no signal telling you it is owed.
Gates
pytest 5877 passed / 17 skipped; performance 9 passed serial; cargo 1742 passed
across 6 suites; ruff clean; inventory gate clean with
--declarations.