Skip to content

C-356: 96.1% of a 5×5 neighbourhood's fatalities sit in one cell — intended, and held in place by three files that nothing checks #483

Description

@Polichinel

Not a bug report — the behaviour is intended and ADR-049 shipped correctly. This is about how the intent is held in place, and about a decision whose justification is expiring.

Everything needed to work this issue is in the repository. No private context, no prior session. Paths below are all tracked.

What was measured, and how to reproduce it

Read from the served store on 2026-09-02:

cell 148759  (13.25N, 39.25E — Mekelle, Tigray, Ethiopia)
  peak month                      121,915
  sum 2020-11 .. 2022-12          273,035
  5x5 neighbourhood sum           284,062
  share held by this ONE cell       96.1%

For scale: across the whole PGM panel the 99.99th percentile is 255 and the median is 0.

Reproduce it yourself (needs ~/.netrc for the data server — see docs/guides/credential_setup.md):

import xarray as xr, numpy as np
from datafactory_query.defaults import DEFAULT_REMOTE
from datafactory_query.backends_zarr import _resolve_storage_options

url = DEFAULT_REMOTE.zarr_url
ds = xr.open_zarr(url, storage_options=_resolve_storage_options(url), consolidated=True)

lat_i = int(round((13.25 + 89.75) / 0.5))   # 206
lon_i = int(round((39.25 + 179.75) / 0.5))  # 438
cell = ds["ged_sb_best"].isel(lat=lat_i, lon=lon_i).sel(time=slice("2020-11", "2022-12")).values
nb   = ds["ged_sb_best"].isel(lat=slice(lat_i-2, lat_i+3),
                              lon=slice(lon_i-2, lon_i+3)).sel(time=slice("2020-11", "2022-12")).values
print(np.nanmax(cell), np.nansum(cell), np.nansum(nb), np.nansum(cell) / np.nansum(nb))

The peak-month figure matches the raw value in
reports/2026-06-28_ucdp_spatial_distribution_of_low_precision_summary_events.md to the digit, three months later, read from the store the models train on.

Cause, not a defect. UCDP records events with where_prec >= 4 — location known only to admin-1 or coarser — and must still attach one lat/lon, so it uses a regional centroid. An entire war's toll lands on one grid point. Systemic rather than one bad record: cell 150919 is the 1999–2000 Eritrea–Ethiopia border war by the same mechanism, twenty years earlier.

ADR-049 shipped exactly as it recommended

docs/ADRs/049_spatial_distribution_of_imprecise_ucdp_events.md asked for the mirror strategy to ship opt-in, with parity as the default. It did:

File What it says
src/datafactory_viewpoint/spatial_distribution.py registers proportional and passthrough
src/datafactory_viewpoint/viewpoint_config.py class default = proportional — the fix, ON
src/datafactory_viewpoint/profiles.py production_parity overrides to passthrough — parity, OFF
scripts/refresh_pipeline.sh calls build_viewpoint.py with no --profile, so the parity profile wins

Nothing here is broken. passthrough's own docstring says it plainly: "Legacy / production-parity behavior: no spatial distribution."

Ask 1 — assert the effective default (small, mechanical)

Read viewpoint_config.py alone and you would conclude the fix is on. It is off only because a profile overrides it, and the CLI defaults to that profile, and the pipeline passes no flag. Three independent defaults must stay aligned, and nothing asserts they do.

Flip any one and delivered values change silently — including the artifact digest that views-postprocessing's test_gaul_lookup_fidelity.py pins, which FAO is served against.

There is an exact precedent to copy. tests/test_falsification_adr049_completeness_r2.py::TestG5GaulCrosswalkPathAlignment already does this shape of check — "the config default must match what the pipeline actually does" — for the GAUL crosswalk paths. The new test is the same idea for the strategy:

  • Add a test beside TestG5GaulCrosswalkPathAlignment asserting the production-effective spatial strategy is passthrough — i.e. that refresh_pipeline.sh invokes build_viewpoint.py in a way that selects the production_parity profile, and that the profile sets spatial_distribution_strategy="passthrough". It should fail if any of the three files drifts.

The point is not to lock passthrough in forever. It is to make a change to any of the three loud rather than silent.

Ask 2 — decide whether parity survives viewser's retirement (a decision, not a task)

Parity exists to reproduce viewser during migration. Viewser is being retired. When it goes, passthrough stops being a deliberate choice and becomes an unexamined one — and ADR-049 already recommends proportional on the merits.

  • Decide whether parity remains the default once viewser retires, and record the decision either way — including "keep parity", which is a legitimate answer that should be written down rather than defaulted into.

Deliberately NOT proposed here

Do not flip the default as part of this issue. It changes every delivered UCDP fatality value in low-precision regions and moves the digest FAO is served against — the same blast radius as #471, and the same reason that one is blocked rather than done. If Ask 2 concludes "flip it", that needs its own change with the consumer coordination that implies.

Background

The where_prec concentration itself had no risk-register entry until 2026-09-02, though ADR-049's documentation drift was registered and resolved back in June. The small half was tracked; the quarter-of-a-million-deaths half was not. Now tracked as C-356 (Tier 3) in reports/technical_risk_register.md, with C-357 (Tier 4) recording a related asymmetry: the same parity constraint gave UCDP a swappable strategy seam that ACLED never got.

Found by following a low-confidence edge in a knowledge graph between two rationale statements — "clean-room design: no legacy system to match" (ADR-028) and "VIEWSER GedLoader parity as the primary constraint" (ADR-023). No single document connects the design stance to its cost: ADR-023 holds the rationale, temporal_distribution.py holds the half that was fixed, and the June report holds the measurement.

See also: ADR-023 (viewpoint builder invariants), ADR-028 (ACLED consolidation), ADR-040 (count conservation — any future spread must preserve it), #471 (the other blocked-on-consumer-coordination fix).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions