Repository navigation
docs(ops): paging woke the phone; close #377 and publish sev1-paging - #418
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aa7e6c8a81
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - **WHEN** the database check becomes critical | ||
| - **THEN** a Pushover channel receives a priority-2 message with `retry` and | ||
| `expire` set |
There was a problem hiding this comment.
Make database-outage paging independent of the failed database
When PostgreSQL is actually unavailable, this scenario cannot deliver the promised Pushover page: _db_check records a critical result, but the later unguarded database queries in _scheduler_check make run_operational_checks raise; run_once then builds an evaluation_failed report and dispatch_event itself raises while querying AlertChannel from the same unavailable database. The only recorded exercise used the channel's test action, so it does not validate this production path; a real database outage falls back to the dead-man page after its grace period instead of sending this priority-2 message.
Useful? React with 👍 / 👎.
Both paths passed on the operator's phone on 2026-09-29: - The Pushover channel's test (21:36 UTC) rang through Do Not Disturb, repeated, and was acknowledged. - With the monitor switched off (UKIP_OPS_MONITOR_ENABLED=0; the app kept serving), Healthchecks.io paged after 15 minutes, the 5-minute period plus 10-minute grace, again through Do Not Disturb. Re-enabling the monitor brought the check back up within one cycle. The runbook suggested stopping the container; switching the monitor off sends Healthchecks.io the same silence without taking the API down. Recorded in the tabletop's corrective action status, whose other rows were stale too: #375, #376 and #378 closed on 2026-09-24, with the note that addresses name the client only from 2026-09-29 (#413). Plan §11 gap 2 is closed and §4 says paging is live; what remains is one person receiving every page (gap 1) and no paging on access (gap 6). The privacy controls overview and the Mexico annex no longer list paging as pending. sev1-paging is archived with all tasks done, and its five requirements are published as openspec/specs/sev1-paging. Closes #377. Co-Authored-By: Claude Opus 5.5 <[email protected]>
aa7e6c8 to
42b1cc6
Compare
Closes #377.
Evidence (2026-09-29, operator's phone)
UKIP_OPS_MONITOR_ENABLED=0, the app kept serving)The runbook suggested stopping the container. Switching the monitor off sends Healthchecks.io the same silence without taking the API down.
Records updated
INCIDENT_TABLETOP_2026-09-22.md, corrective action status:OpenSpec
sev1-pagingis archived with every task done. Its 5 requirements are published asopenspec/specs/sev1-paging.Checks
test_backup_claim_consistency.py: 6 passed🤖 Generated with Claude Code