Skip to content
10 changes: 5 additions & 5 deletions CONSTRAINTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ Invariants this project holds itself to. Every entry is either currently true
or is a stated blocker; nothing here is aspirational. Ratchets are measured,
not estimated, and the command to re-measure each one is given.

Last measured: more than <!-- metric:tests_collected_floor -->2400<!-- /metric --> tests collected, all of which pass except the deliberate
xfails and the skips that `pytest -rs` lists, and <!-- metric:coverage_percent -->94.3<!-- /metric -->%
line coverage (<!-- metric:coverage_lines_hit -->10033<!-- /metric -->/<!-- metric:coverage_lines_total -->10638<!-- /metric --> executable lines) on **CPython <!-- metric:coverage_python -->3.13<!-- /metric -->**, measured
Last measured: more than <!-- metric:tests_collected_floor -->2500<!-- /metric --> tests collected, all of which pass except the deliberate
xfails and the skips that `pytest -rs` lists, and <!-- metric:coverage_percent -->94.4<!-- /metric -->%
line coverage (<!-- metric:coverage_lines_hit -->15023<!-- /metric -->/<!-- metric:coverage_lines_total -->15912<!-- /metric --> executable lines) on **CPython <!-- metric:coverage_python -->3.13<!-- /metric -->**, measured
with `python scripts/coverage_report.py`. The figures come from `docs/data/metrics.json`, which `scripts/metrics.py` checks in CI. Coverage is quoted from that script alone:
it is the project's dependency-free tracer (S1 forbids adding `pytest-cov`), and
a number taken from a different tool is not comparable with the module figures
Expand Down Expand Up @@ -91,9 +91,9 @@ the floors themselves are enforced by `scripts/coverage_report.py --ratchet`.

| Ratchet | Floor | Current (CPython 3.13) | Re-measure with |
|---|---:|---:|---|
| Tests collected | 405 | more than <!-- metric:tests_collected_floor -->2400<!-- /metric --> | `pytest`, `scripts/metrics.py --check` |
| Tests collected | 405 | more than <!-- metric:tests_collected_floor -->2500<!-- /metric --> | `pytest`, `scripts/metrics.py --check` |
| Tests passing | 300 | all collected, except the deliberate xfails and the listed skips | `pytest -rs` |
| Line coverage | 75% | <!-- metric:coverage_percent -->94.3<!-- /metric -->% | `scripts/coverage_report.py`, `scripts/metrics.py --check-coverage` |
| Line coverage | 75% | <!-- metric:coverage_percent -->94.4<!-- /metric -->% | `scripts/coverage_report.py`, `scripts/metrics.py --check-coverage` |
| `io/trajectory.py` coverage | 85% | 95.0% | as above |
| `analysis/findings.py` coverage | 80% | 99.7% | as above |
| `config.py` coverage | 80% | 98.4% | as above |
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ mypy src --ignore-missing-imports
`ruff format src tests scripts` applies formatting; the `--check` form is the
gate.

The suite collects more than <!-- metric:tests_collected_floor -->2400<!-- /metric --> tests and all of them pass on a fresh clone, except the deliberate
The suite collects more than <!-- metric:tests_collected_floor -->2500<!-- /metric --> tests and all of them pass on a fresh clone, except the deliberate
xfails and the skips. `pytest -rs` lists the skips: two need TUM VI reference data that is not
vendored (see below), and the doc-table check runs once `python scripts/run_benchmark.py` has
written `results/benchmark.json`. The count and the other numbers the docs quote are in
Expand Down
2 changes: 1 addition & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Status markers: `[x]` done, `[~]` in progress, `[ ]` not started, `[!]`
blocked. Checked items record what was measured when the item closed. They are not kept current: the
current values are in the README tables, `docs/results.md` and `CONSTRAINTS.md` (B1).

Measured baseline: more than <!-- metric:tests_collected_floor -->2400<!-- /metric --> tests collected and <!-- metric:coverage_percent -->94.3<!-- /metric -->% line coverage on CPython
Measured baseline: more than <!-- metric:tests_collected_floor -->2500<!-- /metric --> tests collected and <!-- metric:coverage_percent -->94.4<!-- /metric -->% line coverage on CPython
<!-- metric:coverage_python -->3.13<!-- /metric --> (`docs/data/metrics.json`, from `scripts/metrics.py` and `scripts/coverage_report.py`). Every
number below re-measures against that, not against a remembered value.

Expand Down
26 changes: 24 additions & 2 deletions docs/adr/0026-urbannav-as-a-real-gnss-dataset.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,9 @@ and is labelled as not blind. Both variants agree on the outage. One sequence, o

## What this does not settle

- The run is one sequence, one receiver and five start perturbations, not five independent tests. Deep-Urban-1, the
planned unseen check, has not been fetched or run.
- The run is one sequence, one receiver and five start perturbations, not five independent tests. Deep-Urban-1 has
since been run as an aided run, and its section of the page says what that does and does not show (see the dated
section below).
- The reference inside the outage is an inertial estimate of unknown error.
- The start attitude needs the rotation between the NovAtel and Xsens units, which is not in the files.
- The F9P height is biased differently on the two sides of the outage, so a run would use the horizontal axes or declare
Expand Down Expand Up @@ -124,3 +125,24 @@ not an outage sequence. The run for it is the aided run over the whole sequence:
no oracle diagnostics (they act only inside an outage and are reported as not applicable), and quantities that do not
need an outage: the normalised errors, the pooled 2-sigma coverage per axis, the gate's rejection rate and the fixes'
own errors against the reference. Outages imposed on this sequence would be a separate experiment, with its own record.

## Added 2026-10-24: the Deep-Urban-1 aided run

This section is dated context. It does not change the Decision section above.

Option A of the section above (the text reference only, no INSPVAX reader) was used, as the maintainer chose. The
receiver file reports a valid fix at every epoch, so the outage rule finds no outage, and the rule was not changed.
The run is the aided run over the whole sequence, with the declared and per-fix variants, five seeds that perturb
only the start state, and the oracle diagnostics not applicable. The rows are in `docs/data/urbannav_run.csv`, the
data rows in `docs/data/urbannav_data.csv`, and the new `docs/data/urbannav_fix_errors.csv` holds the time behaviour
of the receiver's fix errors for both sequences. Tunnel-1 rows in the first two files are byte-identical to before.

What the run found, in words, with every number on [the UrbanNav page](../urbannav.md): the gate rejects none of the
offered fixes although some are far from the reference; the filter is overconfident against the reference in
position and attitude, and the excess is present in every third of the run; the receiver's fix errors are correlated
over tens of seconds on this sequence, which the filter does not model. The last point is an interpretation. The
filter was not run with a correlated-error model, and the reference is a post-processed estimate whose own error is
not known, so some of the correlated difference could be in the reference.

What it does not settle: it is one sequence, one receiver and one recording, and the seeds are not independent tests.
It says nothing about the filter through an outage on this sequence. Imposed outages would be a separate experiment.
8 changes: 4 additions & 4 deletions docs/data/metrics.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
{
"adr_count": 37,
"coverage_lines_hit": 10033,
"coverage_lines_total": 10638,
"coverage_percent": 94.31,
"coverage_lines_hit": 15023,
"coverage_lines_total": 15912,
"coverage_percent": 94.41,
"coverage_python": "3.13",
"outage_visual_window_coverage_percent": 5.0,
"outage_visual_window_nees": 2783.35,
"tests_collected_floor": 2400
"tests_collected_floor": 2500
}
15 changes: 15 additions & 0 deletions docs/data/urbannav_data.csv
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,18 @@ tunnel1,fix_error_first_valid_after_outage,52.76,m
tunnel1,fix_error_max_first_valid_after_outage,52.76,m
tunnel1,fix_vertical_bias_before_outage,-7.63,m
tunnel1,fix_vertical_bias_after_outage,1.164,m
deepurban1,imu_samples,614782,count
deepurban1,imu_span,1537,s
deepurban1,imu_rate_median,400.5,Hz
deepurban1,imu_longest_gap,9.376,ms
deepurban1,reference_epochs,1539,count
deepurban1,reference_spacing_median,1,s
deepurban1,reference_quality_codes,5,count
deepurban1,clock_offset,0.01,s
deepurban1,clock_offset_grid_step,0.01,s
deepurban1,gnss_epochs,1539,count
deepurban1,gnss_denied_epochs,0,count
deepurban1,gnss_outages,0,count
deepurban1,fix_horizontal_error_median,2.361,m
deepurban1,fix_horizontal_error_p95,7.497,m
deepurban1,fix_horizontal_error_max,13.8,m
51 changes: 51 additions & 0 deletions docs/data/urbannav_fix_errors.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
sequence,quantity,value,unit
tunnel1,fix_error_fixes,250,count
tunnel1,fix_error_lag_tolerance,0.5,s
tunnel1,fix_error_offset_east,-1.234,m
tunnel1,fix_error_offset_north,1.95,m
tunnel1,fix_error_rms_about_offset,10.87,m
tunnel1,fix_error_pairs_lag1s,248,count
tunnel1,fix_error_corr_east_lag1s,0.9405,
tunnel1,fix_error_corr_north_lag1s,0.9908,
tunnel1,fix_error_corr_pooled_lag1s,0.9704,
tunnel1,fix_error_pairs_lag10s,230,count
tunnel1,fix_error_corr_east_lag10s,0.07743,
tunnel1,fix_error_corr_north_lag10s,0.3562,
tunnel1,fix_error_corr_pooled_lag10s,0.2331,
tunnel1,fix_error_pairs_lag30s,190,count
tunnel1,fix_error_corr_east_lag30s,-0.05684,
tunnel1,fix_error_corr_north_lag30s,0.03377,
tunnel1,fix_error_corr_pooled_lag30s,-0.00567,
tunnel1,fix_error_pairs_lag60s,130,count
tunnel1,fix_error_corr_east_lag60s,-0.07971,
tunnel1,fix_error_corr_north_lag60s,-0.1313,
tunnel1,fix_error_corr_pooled_lag60s,-0.1038,
tunnel1,fix_error_ess_east,66.8,count
tunnel1,fix_error_ess_east_truncated,0,flag
tunnel1,fix_error_ess_north,18.33,count
tunnel1,fix_error_ess_north_truncated,0,flag
deepurban1,fix_error_fixes,1539,count
deepurban1,fix_error_lag_tolerance,0.5,s
deepurban1,fix_error_offset_east,0.9633,m
deepurban1,fix_error_offset_north,-1.233,m
deepurban1,fix_error_rms_about_offset,3.508,m
deepurban1,fix_error_pairs_lag1s,1538,count
deepurban1,fix_error_corr_east_lag1s,0.9983,
deepurban1,fix_error_corr_north_lag1s,0.9985,
deepurban1,fix_error_corr_pooled_lag1s,0.9984,
deepurban1,fix_error_pairs_lag10s,1529,count
deepurban1,fix_error_corr_east_lag10s,0.9455,
deepurban1,fix_error_corr_north_lag10s,0.95,
deepurban1,fix_error_corr_pooled_lag10s,0.9477,
deepurban1,fix_error_pairs_lag30s,1509,count
deepurban1,fix_error_corr_east_lag30s,0.7442,
deepurban1,fix_error_corr_north_lag30s,0.777,
deepurban1,fix_error_corr_pooled_lag30s,0.7599,
deepurban1,fix_error_pairs_lag60s,1479,count
deepurban1,fix_error_corr_east_lag60s,0.4217,
deepurban1,fix_error_corr_north_lag60s,0.5265,
deepurban1,fix_error_corr_pooled_lag60s,0.4719,
deepurban1,fix_error_ess_east,16.46,count
deepurban1,fix_error_ess_east_truncated,0,flag
deepurban1,fix_error_ess_north,17.87,count
deepurban1,fix_error_ess_north_truncated,0,flag
Loading
Loading