[Telschow]

Part of #44. Part of #113.

Outcome: the candidate is recorded as FAILED ON DEVELOPMENT SEEDS. No point of the grid passed the gate in both scenarios on seeds 0 to 24, so none was chosen, none was confirmed, nothing is frozen, no held-out block is used, and no default changed. Per ADR-0034 the candidate is not repaired and no grid point is added. Base: origin/main at 7d078ed (PR #173), whose CI run was green before the grid was started. Seeds used: 0 to 99 only.

Context

ADR-0034 fixes how tau and rho of the Gauss-Markov visual error model are chosen: a grid of twelve points, the candidate gate of ADR-0030 in S1 and S2 on seeds 0 to 24, a score that is the smallest margin, confirmation on three more blocks. This PR is that run, once, with the code and tests that make it checkable.

Change

• src/navkit/gm_tuning.py and navkit gate tune-gm --csv PATH [--page PATH] [--robustness-csv PATH]. The grid constant is named after the ADR (GRID_TAU_S, GRID_RHO, GRID). The command takes no seed option, and any seed from 100 up is refused before a case runs, in every stage, the diagnostic included.
• The margin functions, the seed check and the seed blocks of the earlier tuning run are imported from navkit.scale_tuning, not copied; a test asserts identity. That module is not touched, so the adaptive-scale CSV and page are unchanged (git diff on scale_tuning.py, its CSV and the golden file is empty). The point, the tie rule (smaller tau, then smaller rho) and the CSV columns differ, so those are new.
• docs/data/gm_tuning.csv (stage, tau, rho, scenario, seeds, condition, relation, value, threshold, margin, holds, score, qualifies, chosen), deterministic, 144 rows. docs/data/gm_tuning_robustness.csv, the diagnostic, 30 rows.
• A generated section in docs/calibration.md: the grid table, the outcome sentence, the diagnostic table. No claim about B1; CONSTRAINTS.md B1 and CLAUDE.md are not touched. tests/test_gm_tuning_page.py is added to the CI documentation step (the guard test requires it).
• The router gets tune-gm. Metric markers are regenerated for the new test count.

The grid (seeds 0 to 24)

Score is the smallest margin over six conditions in both scenarios. Negative means a condition fails.

tau (s)	rho	S1 holds	S2 holds	Score	Smallest margin is	Qualifies
0.25	0.5	yes	no	-7.353	S2 upper	no
0.25	0.75	yes	no	-4.915	S2 upper	no
0.25	0.9	yes	no	-4.17	S2 upper	no
0.5	0.5	no	no	-3.448	S2 upper	no
0.5	0.75	no	no	-2.197	S2 upper	no
0.5	0.9	yes	no	-1.904	S2 upper	no
1	0.5	no	no	-1.814	S2 upper	no
1	0.75	yes	no	-0.9187	S2 upper	no
1	0.9	yes	no	-0.5013	S2 upper	no
2	0.5	no	no	-0.6418	S2 upper	no
2	0.75	yes	no	-0.491	S2 worst_ratio	no
2	0.9	yes	no	-0.4701	S2 worst_ratio	no

The highest score is (tau 2 s, rho 0.9), and it fails S2 on three conditions: worst_ratio 2.205 against 1.5 (margin -0.470, the score), upper 4.985 against 4.738 (-0.052) and coverage 0.9316 against 0.95 (-0.019). It passes S1. Scores rise with tau and rho towards the matching correlation time, but none reaches zero. This is a statement about this grid on these seeds; it does not say why worst_ratio is large.

Confirmation: none. No point was chosen, so no confirmation block was run.

Diagnostic (outside the decision, not a selection input, not a pass or fail)

Point: (tau 2 s, rho 0.9), the point with the highest score because none qualified. The S2-like scenario at five visual-error correlation times, gate of ADR-0030 evaluated for that scenario alone, seeds 25 to 49. Each cell is the margin; "no" marks a condition that does not hold.

Correlation time (s)	upper	lower	coverage	better	median_ratio	worst_ratio	All six hold
0	0.609	-0.290	0.052	0.250	0.604	0.552	no
0.25	0.437	-0.250	0.050	0.100	0.523	0.138	no
0.5	0.343	-0.140	0.047	0.100	0.408	-0.096	no
1	0.238	0.150	0.038	0.000	0.368	-0.444	no
2	0.035	0.473	0.024	-0.050	0.327	-0.860	no

Two things stand out, as observations on development seeds. With little correlation the model is conservative: the lower bound on the per-run NEES fails at 0, 0.25 and 0.5 s. With the correlation time it was set for (2 s) the NEES conditions hold, and the failures are the ATE conditions: the worst ATE ratio against the no-vision control is 2.79 against 1.5, and the candidate beats the control in 19 runs of 25 where 20 are needed. So on seeds 25 to 49 this point would also not have confirmed. That is a diagnostic, not a confirmation, and it was not used to choose anything.

Runtime

One run of the full procedure: 801.8 s (13.4 min) on one laptop, one process. The estimate made first, from timing 5 seeds, was 26 min (18 + 4.5 + 3.7). It was an overestimate because no point qualified, so the 4.5 min of confirmation was not spent.

The grid was run once. My first launch used python -m navkit.cli, which has no main guard and so did nothing: exit 0, no output, no CSV, no case run (checked: no file written, page unchanged). It was a launch error, not a run, and the logic and grid were not touched. The second launch, through the installed navkit command, is the only run.

How tested

ruff check, ruff format --check, mypy src --ignore-missing-imports, check_doc_tables.py (benchmark in a scratch path), metrics.py --check (after --write), check_links.py, mkdocs build --strict (scratch directory), tests/test_framing.py and tests/test_quoted_numbers.py after git add, the golden test without --update (nothing frozen, so it must not change and does not), and the full suite once: 2371 passed, 32 skipped, 2 xfailed, 1 failed (18 min 40 s). The failure was tests/test_b1_gate_page.py::test_the_prose_gives_no_verdict_on_b1: that guard reads the whole page from the B1 section to "Worked example", which includes this new section, and forbids the phrase "gate passes", which my diagnostic table header used. I renamed the column to "All six hold" in the generator and regenerated the page from the unchanged CSVs (no case was rerun). I did not rerun the whole suite. I reran the 35 test files of the CI documentation step plus test_gm_tuning.py, the golden test and test_scale_tuning.py: 803 passed, 6 skipped.

Tests, written first (tests/test_gm_tuning.py, 68 tests, tests/test_gm_tuning_page.py, 18): both files failed at collection before the module existed. They check: the grid constant equals ADR-0034's twelve points (the page test also reads the ADR sentence); the estimator of every point (clone, model on, no preset, no adaptive scale); the score is the minimum margin and not the maximum; ties go to the smaller tau and then the smaller rho, in that order, and a one-in-1e12 difference is not a tie; a non-qualifying point is never chosen; no qualifying point gives "failed"; a chosen point that fails any one confirmation block gives "failed", for each block; a missing or foreign confirmation is an error; the loop runs each point once, confirms only the chosen point on three blocks, and tries no second candidate; seeds 100 and up are refused at every stage before any case, and the command has no seed or held-out option; the CSV columns, byte determinism and round trip; the diagnostic runs at five correlation times on seeds 25 to 49, uses the highest-score point when none qualified, is labelled DIAGNOSTIC, and cannot change the outcome or the grid rows; the page sections equal the CSVs; the margins, scores and choice are recomputed from the rows without the selection function; the config defaults still read as placeholders. Two tests run real cases on seeds 0 and 1 to check determinism and the wiring to the gate, including the correlation time reaching the scenario.

Mutation checks (each reverted, file compared back byte for byte):

Mutation	Result
tie rule: larger tau wins	3 fail (smaller tau, the order of the keys, the highest-score tie rule)
tie rule: larger rho wins	1 fails (smaller rho)
largest margin instead of the smallest	1 fails (the score test)
allow a seed from 100 up	10 fail (the refusal cases, before-any-case, every-stage, not-a-copy)

My first attempt at the last mutation broke collection instead of the tests, so I redid it; the table is the redo.

Risks

• The result is a failure of this candidate on this fixture, not a finding about the method. Synthetic data, two scenarios, development seeds only.
• The diagnostic is read on seeds 25 to 49, which are also a confirmation block. That is allowed because it feeds no decision, but those seeds are now looked at for this point. It does not affect the outcome here, since no confirmation was needed.
• Not done: the fault matrix for the model, any held-out evaluation, any change to vision_enabled, and any attempt to repair the candidate. The next step is a decision for the maintainer; per ADR-0034 it is not a new grid point.
• ci.yml is edited (one test name added to the documentation step).

Checklist

• ruff, ruff format, mypy, tests (full suite once), doc tables, metrics, links, mkdocs strict
• Every number quoted comes from the CSVs or a command run here; the page tables are generated and test-checked
• No # noqa, no new suppression, no skipped or weakened test (one test skips by design when nothing is frozen)
• Golden unchanged: no candidate was frozen
• No credential, key or personal data in the diff
• Conventional Commit title; one logical change

🤖 Generated with Claude Code