docs(ops): paging woke the phone; close #377 and publish sev1-paging

docs/legal/MEXICO_ANNEX.md

@@ -68,9 +68,9 @@ and maintain administrative, technical, and physical security measures. The
 implemented measures and their evidence are documented in
 [PRIVACY_CONTROLS_OVERVIEW.md](PRIVACY_CONTROLS_OVERVIEW.md), including an
 honest register of measures not yet in place. Of the corrective actions from
-the first incident-response exercise of 2026-09-22, read auditing (#375) and
-attributable request logs (#376) are in place; a paging path is implemented
-(#377) and awaits its installation and test on the operator's phone. Still
+the first incident-response exercise of 2026-09-22, read auditing (#375),
+attributable request logs (#376) and a paging path (#377, tested on the
+operator's phone on 2026-09-29) are in place. Still
 open: retention of request logs beyond the container, an external pentest,
 data residency, and a passing restore drill.
 

docs/legal/PRIVACY_CONTROLS_OVERVIEW.md

@@ -31,7 +31,7 @@ prefer an honest register over an impressive one.
 
 | Open item | Tracking ID | Status |
 |-----------|-------------|--------|
-| Incident response capability (incl. breach-notification SLA backing) | ER-IR-001 | Partial — the plan is published, its authority and 72-hour notification commitment were decided on 2026-09-22, and it was first exercised on 2026-09-22. Of that exercise's corrective actions, read auditing and attributable request logs are in place (#375, #376); paging is implemented but not yet installed and tested (#377); request logs are still not retained beyond the container (plan §11 gap 3, `central-log-retention`) |
+| Incident response capability (incl. breach-notification SLA backing) | ER-IR-001 | Partial — the plan is published, its authority and 72-hour notification commitment were decided on 2026-09-22, and it was first exercised on 2026-09-22. Of that exercise's corrective actions, read auditing, attributable request logs and a paging path are in place and tested (#375, #376, #377); request logs are still not retained beyond the container (plan §11 gap 3, `central-log-retention`) |
 | External penetration test | ER-ASSURE-001 | Open — no third-party assessment report available yet |
 | Data residency commitments | ER-DEP-001 | Open — residency follows hosting region; no contractual commitment defined |
 | A passing isolated restore drill | US-073 / ER-BCP-001 | Pending — provider configured and two backup cycles evidenced; the first isolated restore drill (2026-09-21) is recorded as failed and its dossier was approved on 2026-09-21; a passing drill remains |

docs/operating/INCIDENT_RESPONSE_PLAN.md

@@ -94,10 +94,9 @@ What actually reaches a human today:
 | Dokploy | Deploy failures, container restarts | Its own UI |
 | Customer report | Everything nobody instrumented | Email |
 
-**Known detection gaps.** Paging exists in code (#377) but pages nobody until
-the Pushover channel and the Healthchecks.io check are set up and each has been
-tested on the phone ([PAGING_RUNBOOK.md](PAGING_RUNBOOK.md)); until then a SEV1 at 03:00 still waits
-for someone to look at Slack. Only two checks page, and nothing detects
+**Known detection gaps.** Paging is live since 2026-09-29, and both paths have
+woken the phone in a test ([PAGING_RUNBOOK.md](PAGING_RUNBOOK.md)). One person
+receives every page (gap 1). Only two checks page, and nothing detects
 unauthorized access on its own (gap 6). No central log retention: the backend's
 container log is ephemeral and disappears with the container, which is why
 capturing it is the first evidence step below; Traefik's access log survives
@@ -338,14 +337,14 @@ Recorded here so nobody discovers them mid-incident:
    Accepted by the owner on 2026-09-22 as what the current team can commit to,
    and revisited when funding and staffing allow a second responder. It is also
    why the notification commitment is 72 hours rather than 24.
-2. **No paging** (#377). Alerts reach Slack and the container log; nothing
-   wakes anyone up. Measured in the 2026-09-22 tabletop: **6 h 22 min** from
-   exposure to detection, overnight, and the report came from a third party.
-   **In code since 2026-09-26** (§4): Pushover emergency pages for `database`
-   and `migrations` critical, and a Healthchecks.io dead man's switch for the
-   monitor itself. It stays open until both are installed and have woken the
-   phone in a test. What remains after that is narrower: one person receives
-   every page (gap 1), and only availability pages, not access (gap 6).
+2. ~~**No paging** (#377).~~ **Closed 2026-09-29.** The 2026-09-22 tabletop
+   measured **6 h 22 min** from exposure to detection, overnight, reported by a
+   third party. Now `database` and `migrations` critical page through Pushover
+   at emergency priority, and a Healthchecks.io dead man's switch pages when
+   the monitor itself goes silent (§4). Both woke the phone through Do Not
+   Disturb in a test (INCIDENT_TABLETOP_2026-09-22.md, corrective action
+   status). What remains is narrower: one person receives every page (gap 1),
+   and only availability pages, not access (gap 6).
 3. **No central log retention.** The backend's container log is ephemeral, so
    early capture is the only way to keep it, and for ordinary reads outside the
    audited classes (gap 9) it is the only record that the read happened at all.

docs/operating/INCIDENT_TABLETOP_2026-09-22.md

@@ -100,10 +100,10 @@ Added after the exercise; the record above is left as it was written.
 | Gap | Status |
 |---|---|
 | Per-session revocation (#368 phase C.3) | **Closed 2026-09-23.** Tokens carry a `sid` naming their session, every request resolves the user through it, and a session can be revoked alone — including the operator's own, which deactivation refuses. Plan §5.3 |
-| Reads are not audited (#375) | Open |
-| Request log has no identity (#376) | Open |
-| Nothing pages (#377) | Open |
-| No `ip_address` filter on the audit log (#378) | Open |
+| Reads are not audited (#375) | **Closed 2026-09-24** (#395). Exports, audit-log reads, API-key reads and bulk reads are audited with their session or key. Ordinary first-page reads are not, by design (plan §7) |
+| Request log has no identity (#376) | **Closed 2026-09-24** (#387). Each authenticated line names the user and the session or API key. Its `client_ip` names the client only since 2026-09-29 (#413) |
+| Nothing pages (#377) | **Closed 2026-09-29.** Both paths woke the operator's phone on 2026-09-29: the Pushover channel's test at 21:36 UTC rang through Do Not Disturb, repeated and was acknowledged; with the monitor switched off (`UKIP_OPS_MONITOR_ENABLED=0`, the app kept serving), Healthchecks.io paged after 15 minutes (5-minute period plus 10-minute grace), again through Do Not Disturb (reported 22:29 UTC). Re-enabling the monitor brought the check back up within one cycle. |
+| No `ip_address` filter on the audit log (#378) | **Closed 2026-09-24** (#384). The addresses it filters on are the client's only from 2026-09-29 03:57 UTC (#413); earlier rows carry a proxy's |
 | Blast radius undeterminable (ER-BCP-001 residual risk 1) | Open |
 
 ## What was verified to work

openspec/changes/sev1-paging/tasks.md → openspec/changes/archive/2026-09-29-sev1-paging/tasks.md

@@ -59,6 +59,16 @@ TDD throughout. Sections 1–3 are backend and can ship as one PR; 4 is the UI;
       to `ops.check_failed`, create the Healthchecks.io check (period = monitor
       interval, grace = 2 intervals, Pushover integration at emergency), set
       `UKIP_OPS_HEARTBEAT_URL`.
-- [ ] 5.3 **Operator:** test the channel with the phone in Do Not Disturb and
+- [x] 5.3 **Operator:** test the channel with the phone in Do Not Disturb and
       confirm it rings; stop the backend container and confirm Healthchecks.io
       pages; record both in the tabletop follow-up. Then close #377.
+      Both paths woke the operator's phone on 2026-09-29: the Pushover
+      channel's test at 21:36 UTC rang through Do Not Disturb, repeated
+      and was acknowledged; with the monitor switched off
+      (`UKIP_OPS_MONITOR_ENABLED=0`, the app kept serving),
+      Healthchecks.io paged after 15 minutes (5-minute period plus
+      10-minute grace), again through Do Not Disturb (reported 22:29 UTC).
+      Re-enabling the monitor brought the check back up within one cycle.
+      The heartbeat test switched the monitor off instead of stopping the
+      container: the same silence reaches Healthchecks.io, and the app stayed
+      up. Recorded in `INCIDENT_TABLETOP_2026-09-22.md`.

openspec/specs/sev1-paging/spec.md

@@ -0,0 +1,82 @@
+# sev1-paging Specification
+
+## Purpose
+A SEV1 wakes a person: ops alerts carry a severity derived from which checks are critical, Pushover pages at emergency priority through Do Not Disturb once per incident, and a Healthchecks.io dead man's switch pages when the monitor itself goes silent. Created by archiving change sev1-paging.
+## Requirements
+### Requirement: Pushover is an alert channel type
+
+The system SHALL accept `pushover` as an alert channel type, store its
+application token and user key encrypted, never return them, and deliver alerts
+to Pushover's messages API without raising on failure.
+
+#### Scenario: Credentials are validated and never returned
+
+- **WHEN** an admin creates a `pushover` channel without a user key
+- **THEN** the request is rejected with 422
+- **AND WHEN** the channel is created with both
+- **THEN** no response ever contains the token or the user key
+
+### Requirement: Ops alerts carry a severity from the critical checks
+
+The system SHALL classify each ops alert as `page` when `database` or
+`migrations` is critical, `urgent` when `backup_freshness` or `secrets` is
+critical (and no paging check is), and `info` otherwise, and SHALL map them to
+Pushover priorities 2, 1 and 0.
+
+#### Scenario: The database going down pages
+
+- **WHEN** the database check becomes critical
+- **THEN** a Pushover channel receives a priority-2 message with `retry` and
+  `expire` set
+
+#### Scenario: A stale backup does not wake anyone
+
+- **WHEN** only `backup_freshness` is critical
+- **THEN** the Pushover message has priority 1
+
+#### Scenario: Slack sees the same classification
+
+- **WHEN** an alert is `page`
+- **THEN** its Slack message starts with `[PAGE]`
+
+### Requirement: A page is sent once per incident
+
+The system SHALL send severity `page` only when the set of critical paging
+checks grows relative to the previous observation, and SHALL send reminders and
+unrelated changes during that incident as `info`.
+
+#### Scenario: A reminder during an outage does not page again
+
+- **WHEN** the database has been critical for longer than the reminder interval
+- **THEN** the reminder is sent at priority 0
+
+#### Scenario: A second paging check failing pages again
+
+- **WHEN** migrations become critical while the database already was
+- **THEN** a new priority-2 page is sent
+
+### Requirement: The monitor reports that it is alive
+
+The system SHALL, when `UKIP_OPS_HEARTBEAT_URL` is set, ping it after every
+monitor cycle, ping `<url>/fail` when the evaluation raised, never raise, and
+never log the URL.
+
+#### Scenario: A failed evaluation reports failure
+
+- **WHEN** the checks raise during a cycle
+- **THEN** the heartbeat request goes to the `/fail` path
+
+#### Scenario: An unreachable heartbeat service does not break the cycle
+
+- **WHEN** the heartbeat request times out
+- **THEN** the cycle completes and the next one runs
+
+### Requirement: The paging path is testable end to end
+
+The system SHALL, for a Pushover channel's test action, send a real
+emergency-priority message that expires on its own within a minute.
+
+#### Scenario: Testing a Pushover channel
+
+- **WHEN** an admin tests a Pushover channel
+- **THEN** a priority-2 message with `expire` of 60 seconds is sent