Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,12 @@ this project uses [semantic versioning](https://semver.org/).
The Track A exit test is not met. [ADR-0021](docs/adr/0021-fault-classification-from-innovations.md) is accepted:
ship a multipath detector at most, and report every other detected fault as cause unknown.

- `navkit.fdir.multipath` and [`docs/multipath.md`](docs/multipath.md): a multipath signature (NIS and lag-1
autocorrelation of a window of GNSS innovations) in a sensitive and a specific rule, with shipping criteria written
before the fresh runs. The specific rule met them: it called all multipath of 3 m and above and none of the other
runs, and none of 136 controls. The sensitive rule did not (14% of the other runs). It is a function, not wired into
the filter. [ADR-0022](docs/adr/0022-multipath-signature.md) is accepted. `navkit sweep multipath` runs the study.

### Changed

- Blocker B4 is resolved by retiring the claim. The two tests that compared a published Basalt estimate of TUM VI
Expand Down
12 changes: 6 additions & 6 deletions CONSTRAINTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ Invariants this project holds itself to. Every entry is either currently true
or is a stated blocker; nothing here is aspirational. Ratchets are measured,
not estimated, and the command to re-measure each one is given.

Last measured: <!-- metric:tests_collected -->1245<!-- /metric --> tests collected, all of which pass except the deliberate
xfails and the skips that `pytest -rs` lists, and <!-- metric:coverage_percent -->94.1<!-- /metric -->%
line coverage (<!-- metric:coverage_lines_hit -->8708<!-- /metric -->/<!-- metric:coverage_lines_total -->9253<!-- /metric --> executable lines) on **CPython <!-- metric:coverage_python -->3.13<!-- /metric -->**, measured
Last measured: <!-- metric:tests_collected -->1259<!-- /metric --> tests collected, all of which pass except the deliberate
xfails and the skips that `pytest -rs` lists, and <!-- metric:coverage_percent -->94.2<!-- /metric -->%
line coverage (<!-- metric:coverage_lines_hit -->9039<!-- /metric -->/<!-- metric:coverage_lines_total -->9598<!-- /metric --> executable lines) on **CPython <!-- metric:coverage_python -->3.13<!-- /metric -->**, measured
with `python scripts/coverage_report.py`. The figures come from `docs/data/metrics.json`, which `scripts/metrics.py` checks in CI. Coverage is quoted from that script alone:
it is the project's dependency-free tracer (S1 forbids adding `pytest-cov`), and
a number taken from a different tool is not comparable with the module figures
Expand Down Expand Up @@ -83,9 +83,9 @@ These are floors, not goals. Each must not regress; raising one is welcome.

| Ratchet | Floor | Current (CPython 3.13) | Re-measure with |
|---|---:|---:|---|
| Tests collected | 405 | <!-- metric:tests_collected -->1245<!-- /metric --> | `pytest`, `scripts/metrics.py --check` |
| Tests collected | 405 | <!-- metric:tests_collected -->1259<!-- /metric --> | `pytest`, `scripts/metrics.py --check` |
| Tests passing | 300 | all collected, except the deliberate xfails and the listed skips | `pytest -rs` |
| Line coverage | 75% | <!-- metric:coverage_percent -->94.1<!-- /metric -->% | `scripts/coverage_report.py`, `scripts/metrics.py --check-coverage` |
| Line coverage | 75% | <!-- metric:coverage_percent -->94.2<!-- /metric -->% | `scripts/coverage_report.py`, `scripts/metrics.py --check-coverage` |
| `io/trajectory.py` coverage | 85% | 95.0% | as above |
| `analysis/findings.py` coverage | 80% | 99.7% | as above |
| `config.py` coverage | 80% | 98.4% | as above |
Expand All @@ -104,7 +104,7 @@ These are floors, not goals. Each must not regress; raising one is welcome.
| `outage_sweep.py` coverage | 80% | 98.4% | as above |
| `fdir/nis_monitor.py` coverage | 80% | 98.8% | as above |
| `degrade/` coverage | 50% | 78.0% (`config.py`) / 84.2% (`inject.py`) | as above |
| Docs | README + architecture + calibration + every ADR | <!-- metric:adr_count -->21<!-- /metric --> ADRs, all in the index | `scripts/metrics.py --check` |
| Docs | README + architecture + calibration + every ADR | <!-- metric:adr_count -->22<!-- /metric --> ADRs, all in the index | `scripts/metrics.py --check` |
| Documented tables match the generated benchmark | exact | yes | `scripts/check_doc_tables.py` |
| Open blockers documented | all | see ROADMAP | manual |

Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ mypy src --ignore-missing-imports
`ruff format src tests scripts` applies formatting; the `--check` form is the
gate.

The suite collects <!-- metric:tests_collected -->1245<!-- /metric --> tests and all of them pass on a fresh clone, except the deliberate
The suite collects <!-- metric:tests_collected -->1259<!-- /metric --> tests and all of them pass on a fresh clone, except the deliberate
xfails and the skips. `pytest -rs` lists the skips: two need TUM VI reference data that is not
vendored (see below), and the doc-table check runs once `python scripts/run_benchmark.py` has
written `results/benchmark.json`. The count and the other numbers the docs quote are in
Expand Down
4 changes: 3 additions & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Sequenced by dependency, not by ambition. Each item names its exit test.
Status markers: `[x]` done, `[~]` in progress, `[ ]` not started, `[!]`
blocked.

Measured baseline: <!-- metric:tests_collected -->1245<!-- /metric --> tests collected and <!-- metric:coverage_percent -->94.1<!-- /metric -->% line coverage on CPython
Measured baseline: <!-- metric:tests_collected -->1259<!-- /metric --> tests collected and <!-- metric:coverage_percent -->94.2<!-- /metric -->% line coverage on CPython
<!-- metric:coverage_python -->3.13<!-- /metric --> (`docs/data/metrics.json`, from `scripts/metrics.py` and `scripts/coverage_report.py`). Every
number below re-measures against that, not against a remembered value.

Expand Down Expand Up @@ -203,6 +203,8 @@ that has already been wrong. This track moves detection forward.
- [~] Per-sensor classification from the GNSS innovations is measured ([fault classification](docs/fault_classification.md),
ADR-0021 accepted): multipath is separable, spoofing is called in about three quarters of runs, weak faults are
missed, and the control false-alarm rate is 18% in cross-validation and 35% on unseen runs.
- [x] Multipath signature as a function, not wired into the filter ([multipath](docs/multipath.md), ADR-0022): the
specific rule calls multipath of about 3 m and above on injected noise and nothing else on fresh runs.
- [ ] Per-sensor detection of multipath (elevated innovation variance without
a mean shift), spoofing (innovation consistent but GNSS-internally
inconsistent, e.g. against the IMU-predicted position), and sensor
Expand Down
33 changes: 33 additions & 0 deletions docs/adr/0022-multipath-signature.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# ADR-0022: A multipath signature on the GNSS innovations

- Status: accepted
- Date: 2026-10
- Related: [ADR-0021](0021-fault-classification-from-innovations.md), [ADR-0005](0005-chi-square-fdir-gating.md), [multipath](../multipath.md)

## Context

ADR-0021 chose to ship at most a multipath detector. A rule of two numbers on a window of GNSS innovations (NIS and
lag-1 autocorrelation) was tested in two versions, with the shipping criteria written before the fresh runs.

## Options

1. **The sensitive rule** (NIS at least 2). It sees multipath from 1 m and calls small steps multipath too.
2. **The specific rule** (NIS at least 6). It sees multipath from about 3 m and called nothing else on the fresh runs.
3. **Wire a detector into the filter.** The filter would report multipath as a cause on the FDIR event.

## Decision

Option 2, as a function and not as filter behaviour. `navkit.fdir.multipath.signature` and `is_multipath` use the
specific rule by default. The sensitive rule stays available as an argument, so the trade-off can be seen. Nothing in
the estimator or the FDIR layer calls it, and no FDIR event changes. Wiring it in (option 3) is a separate step with
its own ADR, because it changes what a user of the filter is told.

The criteria were met on fresh runs by the specific rule only: 100% of the multipath runs of 3 m and above, and none
of the other runs. The floor is about 3 m.

## Consequences

The claim is limited to injected white noise on simulated GNSS, two datasets and 20 s windows. Real multipath is
correlated and may exceed the autocorrelation limit. A real-receiver check is not done. The Track A exit test is
still not met: the multipath call alone does not give 95% detection per type at zero false alarms for spoofing and
degradation.
1 change: 1 addition & 0 deletions docs/adr/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,4 @@ what it cost, and what was rejected.
| [ADR-0019](0019-slow-ramp-spoofing.md) | Slow-ramp GNSS spoofing is not detectable with IMU and GNSS alone | accepted |
| [ADR-0020](0020-second-source-against-a-ramp.md) | A second source against a slow-ramp spoof | accepted |
| [ADR-0021](0021-fault-classification-from-innovations.md) | Fault classification from the GNSS innovations | accepted |
| [ADR-0022](0022-multipath-signature.md) | A multipath signature on the GNSS innovations | accepted |
10 changes: 5 additions & 5 deletions docs/data/metrics.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{
"adr_count": 21,
"coverage_lines_hit": 8708,
"coverage_lines_total": 9253,
"coverage_percent": 94.11,
"adr_count": 22,
"coverage_lines_hit": 9039,
"coverage_lines_total": 9598,
"coverage_percent": 94.18,
"coverage_python": "3.13",
"tests_collected": 1245
"tests_collected": 1259
}
Loading
Loading