Skip to content

Add the stability report between runs (S8) - #11

Closed
mgilardi wants to merge 1 commit into
mainfrom
s8-stability
Closed

mgilardi wants to merge 1 commit into
mainfrom
s8-stability

Conversation

@mgilardi

Copy link
Copy Markdown
Contributor

Closes K408. compareRuns / assertStable in onefamily-analysis: labels matched by overlap; stop when more than 5% of people placed in both runs change line at two levels; deeper levels, groups, newly placed and no longer placed reported. 'Too few' (under 50) is reported, not a stop, like S7's side check.

Tested: 282 tests, typecheck, leak check. Real exercise outside the repo: original run vs OneFamily's on the first finished kit, 1.9% changed, stable. Not wired into the app (one run per kit so far).

🤖 Generated with Claude Code

A new run of a kit should leave most people on the line they were on.
compareRuns matches branch labels by overlap, so renumbering isn't
counted as change, and assertStable stops when more than 5% of the people
placed in both runs change line (side and branch). Deeper levels and
groups are reported without stopping, since third-level splits are often
borderline.

Tested on made-up runs and a planted family. On the first finished kit's
real data, outside the repo, the original run against OneFamily's came out
stable at 1.9%. The app keeps one run per kit, so nothing calls it yet.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@mgilardi

Copy link
Copy Markdown
Contributor Author

Replaced by a rebased branch (no force-push).

@mgilardi mgilardi closed this Nov 15, 2026
@mgilardi
mgilardi deleted the s8-stability branch November 15, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant