A real front end's errors and the stochastic clone (WB-3); ADR-0023

CHANGELOG.md

@@ -120,6 +120,14 @@ this project uses [semantic versioning](https://semver.org/).
   runs, and none of 136 controls. The sensitive rule did not (14% of the other runs). It is a function, not wired into
   the filter. [ADR-0022](docs/adr/0022-multipath-signature.md) is accepted. `navkit sweep multipath` runs the study.
 
+- `navkit euroc fetch --images`, `navkit.vo_frontend`, `navkit euroc vo` and [`docs/real_front_end.md`](docs/real_front_end.md):
+  a NumPy two-view feature-tracking front end, measured against the ground truth on 30 s of real EuRoC frames. Its errors
+  are small (0.08° and 8 mm per one-frame pair, with the true scale) and correlated for 0.2 to 0.25 s. A Gaussian
+  surrogate with that size and correlation time keeps the stochastic clone inside its gate. Replaying the measured
+  errors does not (NEES 27, 55% coverage), and assuming four times the measured noise brings it back (NEES 4.3, 91%).
+  One sequence and one window. [ADR-0023](docs/adr/0023-clone-against-a-measured-front-end.md) is accepted: the clone
+  stays opt-in, with the inflation.
+
 ### Changed
 
 - The implementation plan carries the same point-in-time banner as the other baseline records. P5-09 and P5-12 are

CONSTRAINTS.md

@@ -4,9 +4,9 @@ Invariants this project holds itself to. Every entry is either currently true
 or is a stated blocker; nothing here is aspirational. Ratchets are measured,
 not estimated, and the command to re-measure each one is given.
 
-Last measured: <!-- metric:tests_collected -->1259<!-- /metric --> tests collected, all of which pass except the deliberate
-xfails and the skips that `pytest -rs` lists, and <!-- metric:coverage_percent -->94.2<!-- /metric -->%
-line coverage (<!-- metric:coverage_lines_hit -->9039<!-- /metric -->/<!-- metric:coverage_lines_total -->9598<!-- /metric --> executable lines) on **CPython <!-- metric:coverage_python -->3.13<!-- /metric -->**, measured
+Last measured: <!-- metric:tests_collected -->1302<!-- /metric --> tests collected, all of which pass except the deliberate
+xfails and the skips that `pytest -rs` lists, and <!-- metric:coverage_percent -->94.3<!-- /metric -->%
+line coverage (<!-- metric:coverage_lines_hit -->9819<!-- /metric -->/<!-- metric:coverage_lines_total -->10408<!-- /metric --> executable lines) on **CPython <!-- metric:coverage_python -->3.13<!-- /metric -->**, measured
 with `python scripts/coverage_report.py`. The figures come from `docs/data/metrics.json`, which `scripts/metrics.py` checks in CI. Coverage is quoted from that script alone:
 it is the project's dependency-free tracer (S1 forbids adding `pytest-cov`), and
 a number taken from a different tool is not comparable with the module figures
@@ -83,9 +83,9 @@ These are floors, not goals. Each must not regress; raising one is welcome.
 
 | Ratchet | Floor | Current (CPython 3.13) | Re-measure with |
 |---|---:|---:|---|
-| Tests collected | 405 | <!-- metric:tests_collected -->1259<!-- /metric --> | `pytest`, `scripts/metrics.py --check` |
+| Tests collected | 405 | <!-- metric:tests_collected -->1302<!-- /metric --> | `pytest`, `scripts/metrics.py --check` |
 | Tests passing | 300 | all collected, except the deliberate xfails and the listed skips | `pytest -rs` |
-| Line coverage | 75% | <!-- metric:coverage_percent -->94.2<!-- /metric -->% | `scripts/coverage_report.py`, `scripts/metrics.py --check-coverage` |
+| Line coverage | 75% | <!-- metric:coverage_percent -->94.3<!-- /metric -->% | `scripts/coverage_report.py`, `scripts/metrics.py --check-coverage` |
 | `io/trajectory.py` coverage | 85% | 95.0% | as above |
 | `analysis/findings.py` coverage | 80% | 99.7% | as above |
 | `config.py` coverage | 80% | 98.4% | as above |
@@ -104,7 +104,7 @@ These are floors, not goals. Each must not regress; raising one is welcome.
 | `outage_sweep.py` coverage | 80% | 98.4% | as above |
 | `fdir/nis_monitor.py` coverage | 80% | 98.8% | as above |
 | `degrade/` coverage | 50% | 78.0% (`config.py`) / 84.2% (`inject.py`) | as above |
-| Docs | README + architecture + calibration + every ADR | <!-- metric:adr_count -->22<!-- /metric --> ADRs, all in the index | `scripts/metrics.py --check` |
+| Docs | README + architecture + calibration + every ADR | <!-- metric:adr_count -->23<!-- /metric --> ADRs, all in the index | `scripts/metrics.py --check` |
 | Documented tables match the generated benchmark | exact | yes | `scripts/check_doc_tables.py` |
 | Open blockers documented | all | see ROADMAP | manual |
 

CONTRIBUTING.md

@@ -70,7 +70,7 @@ mypy src --ignore-missing-imports
 `ruff format src tests scripts` applies formatting; the `--check` form is the
 gate.
 
-The suite collects <!-- metric:tests_collected -->1259<!-- /metric --> tests and all of them pass on a fresh clone, except the deliberate
+The suite collects <!-- metric:tests_collected -->1302<!-- /metric --> tests and all of them pass on a fresh clone, except the deliberate
 xfails and the skips. `pytest -rs` lists the skips: two need TUM VI reference data that is not
 vendored (see below), and the doc-table check runs once `python scripts/run_benchmark.py` has
 written `results/benchmark.json`. The count and the other numbers the docs quote are in

ROADMAP.md

@@ -4,7 +4,7 @@ Sequenced by dependency, not by ambition. Each item names its exit test.
 Status markers: `[x]` done, `[~]` in progress, `[ ]` not started, `[!]`
 blocked.
 
-Measured baseline: <!-- metric:tests_collected -->1259<!-- /metric --> tests collected and <!-- metric:coverage_percent -->94.2<!-- /metric -->% line coverage on CPython
+Measured baseline: <!-- metric:tests_collected -->1302<!-- /metric --> tests collected and <!-- metric:coverage_percent -->94.3<!-- /metric -->% line coverage on CPython
 <!-- metric:coverage_python -->3.13<!-- /metric --> (`docs/data/metrics.json`, from `scripts/metrics.py` and `scripts/coverage_report.py`). Every
 number below re-measures against that, not against a remembered value.
 
@@ -259,8 +259,10 @@ this removes it.
       needed there.
 - [~] A visual front end with correlated errors, scale ambiguity and outliers.
       Outliers and scale drift are survived. Correlated errors are not, unless the
-      assumed visual noise is inflated by a factor found by trial. A real front end
-      is still untested.
+      assumed visual noise is inflated by a factor found by trial. A plain two-view front end was
+      then measured on real frames ([real front end](docs/real_front_end.md), ADR-0023): its errors are
+      correlated for about 0.25 s, and when they are replayed the clone stays inside the gate only with the assumed
+      noise inflated by four. One sequence and one window, so the default does not change.
 
 **Exit test:** NEES below 10 with vision enabled and GNSS denied, coverage above
 90%, `vision_enabled = True` as the shipped default. Current value 286.2 for the

docs/adr/0023-clone-against-a-measured-front-end.md

@@ -0,0 +1,32 @@
+# ADR-0023: The clone against a measured front end
+
+- Status: accepted
+- Date: 2026-10
+- Related: [ADR-0017](0017-stochastic-clone-for-the-visual-update.md), [ADR-0020](0020-second-source-against-a-ramp.md), [real front end](../real_front_end.md)
+
+## Context
+
+ADR-0017 left the clone opt-in because it is calibrated for independent visual errors, and no data said which a real
+front end has. A plain NumPy two-view front end was run on 30 s of real EuRoC frames and its errors were measured. They
+are small and correlated over a fraction of a second. A Gaussian surrogate with that size and correlation time keeps
+the clone inside its gate. Replaying the measured errors themselves does not: NEES 27 and 55% coverage. Assuming four
+times the measured noise brings the clone back inside, with NEES 4.3 and 91% coverage.
+
+## Options
+
+1. **Leave the clone opt-in and record the finding.** `vision_enabled` stays False. The opt-in clone is used with the
+   assumed visual noise inflated by four.
+2. **Make the clone the default with the inflation.** One sequence and one error series do not support that.
+3. **Measure more first.** More sequences, other windows, and a keyframe-based front end.
+
+## Decision
+
+Options 1 and 3. `vision_enabled` stays False and the clone stays opt-in. For a feature-tracking front end like the
+one measured, the opt-in clone should assume four times the measured visual noise. B1 stays open for the shipped
+configuration. The next step is more sequences and windows, so that the replay result has a spread.
+
+## Consequences
+
+The finding is limited to this front end, this window and a known translation scale. The factor of four is a trial
+value that held in one replay, not a derived constant. A front end with another error structure may need another
+factor.

docs/adr/index.md

@@ -27,3 +27,4 @@ what it cost, and what was rejected.
 | [ADR-0020](0020-second-source-against-a-ramp.md) | A second source against a slow-ramp spoof | accepted |
 | [ADR-0021](0021-fault-classification-from-innovations.md) | Fault classification from the GNSS innovations | accepted |
 | [ADR-0022](0022-multipath-signature.md) | A multipath signature on the GNSS innovations | accepted |
+| [ADR-0023](0023-clone-against-a-measured-front-end.md) | The clone against a measured front end | accepted |

docs/data/metrics.json

@@ -1,8 +1,8 @@
 {
-  "adr_count": 22,
-  "coverage_lines_hit": 9039,
-  "coverage_lines_total": 9598,
-  "coverage_percent": 94.18,
+  "adr_count": 23,
+  "coverage_lines_hit": 9819,
+  "coverage_lines_total": 10408,
+  "coverage_percent": 94.34,
   "coverage_python": "3.13",
-  "tests_collected": 1259
+  "tests_collected": 1302
 }

docs/data/vo_clone_check.csv

@@ -0,0 +1,41 @@
+variant,seed,nees_mean,coverage_2sigma,final_error_m,max_error_m,claimed_sigma_m
+IMU only,0,3.011047321411209,0.9933333333333333,83.70962849938226,83.70962849938226,67.65150263056789
+IMU only,1,1.2360985418027506,1.0,54.45549242201074,54.45549242201074,57.86877385629817
+IMU only,2,1.324263346894136,1.0,55.20144456663034,55.20144456663034,56.15092314604725
+IMU only,3,2.3613986856286706,1.0,80.1061970879712,80.1061970879712,43.960815248386915
+IMU only,4,1.479515039799907,1.0,64.33376427977204,64.33376427977204,48.223981432956414
+"anchor, measured errors",0,114171.9816903814,0.06,186.05326920757645,186.05326920757645,0.2682960633187594
+"anchor, measured errors",1,4582.681034102779,0.17,38.98247257418997,38.98247257418997,0.268296203620487
+"anchor, measured errors",2,152632.04754921602,0.051666666666666666,216.2950405581362,216.2950405581362,0.2682961539560762
+"anchor, measured errors",3,8030.747573807465,0.056666666666666664,50.85960642180771,50.85960642180771,0.2682962188672265
+"anchor, measured errors",4,2220.9189909754346,0.14033333333333334,24.316707443086752,24.316707443086752,0.26829611289389843
+"clone, independent errors",0,4.007167125686769,1.0,0.38716234234971053,0.49027929441335305,0.27359716338666507
+"clone, independent errors",1,2.42816370355988,1.0,0.3133735209951239,1.0208394956966504,0.27361401514532574
+"clone, independent errors",2,2.039420174409159,1.0,0.2080947618626702,1.0644072764533532,0.27362755542902895
+"clone, independent errors",3,1.2618167376906853,1.0,0.30142258936237515,1.8443749062870949,0.27360673913866773
+"clone, independent errors",4,1.2597964597480529,1.0,0.19159394036943125,0.8911346077089927,0.27360208628623983
+"clone, measured correlation",0,6.512898646369463,0.9268333333333333,0.5905588561710103,0.6339310071038039,0.27358432381512754
+"clone, measured correlation",1,3.233643396304074,1.0,0.4542393202660976,1.0208394956966504,0.27362024011269753
+"clone, measured correlation",2,7.488165133087823,0.8908333333333334,0.5081053426657918,1.0648651159764033,0.27361634902765025
+"clone, measured correlation",3,5.586344967445342,0.9285,0.6350638421311661,1.8443749062870949,0.273625792629978
+"clone, measured correlation",4,2.3685209725408223,1.0,0.27962606278823426,0.8911346077089927,0.2735997101295965
+"clone, measured correlation, noise assumed x4",0,1.4577914734607267,1.0,0.4930178167880968,0.5788644786013961,0.6697355615787384
+"clone, measured correlation, noise assumed x4",1,1.4243188387654842,1.0,0.5911548562833633,1.0208394956966504,0.6647157977037044
+"clone, measured correlation, noise assumed x4",2,1.1598820006352226,1.0,0.3853541069446335,1.0644072764533532,0.6663882330282552
+"clone, measured correlation, noise assumed x4",3,1.3663597214021872,1.0,0.8468382431054775,1.8443749062870949,0.6770825779385757
+"clone, measured correlation, noise assumed x4",4,0.9313382116502497,1.0,0.3461272565693066,0.8911346077089927,0.6696683908514616
+"replay, anchor",0,162853.3177202277,0.045,238.5619630685604,238.5619630685604,0.2738550680289662
+"replay, anchor",1,3902.892646131298,0.22133333333333333,40.40266180548525,40.40266180548525,0.27385520648124717
+"replay, anchor",2,191652.38118501462,0.0375,259.8637367849311,259.8637367849311,0.2738551681094766
+"replay, anchor",3,23497.854123016754,0.05,93.63839724696794,93.63839724696794,0.27385521817424435
+"replay, anchor",4,15496.151456115414,0.09766666666666667,66.38652929287211,66.38652929287211,0.2738550878280069
+"replay, clone",0,34.587376134194805,0.378,0.4410105430927355,2.0679822624887816,0.2736007756934708
+"replay, clone",1,26.97402561069759,0.5571666666666667,0.3076255905617748,2.0177121447026427,0.27360843161141396
+"replay, clone",2,30.985558012009587,0.535,0.24979233572188436,2.124069403692931,0.2736292940411207
+"replay, clone",3,26.427569590150807,0.568,0.1168379946950831,1.9991510229137401,0.27359348189691307
+"replay, clone",4,23.426648244921132,0.5501666666666667,0.05654685195852756,1.8729005865056703,0.27360567393794893
+"replay, clone, noise assumed x4",0,4.967967593940472,0.895,0.2982003603348016,2.315065365306037,0.6728826925288418
+"replay, clone, noise assumed x4",1,4.699123780634335,0.8981666666666667,0.8655240935112204,2.315638468206952,0.6703569638405952
+"replay, clone, noise assumed x4",2,4.305557510468603,0.9071666666666667,0.7537660844255686,2.3080299325352716,0.6682262200970166
+"replay, clone, noise assumed x4",3,3.9705814909040975,0.9166666666666666,0.7721493615124901,2.208643976825519,0.6729112733384796
+"replay, clone, noise assumed x4",4,3.393820812165554,0.9923333333333333,0.6021227381077545,2.1003063486798905,0.6691948130013253