Skip to content

docs(ops): paging woke the phone; close #377 and publish sev1-paging - #418

Merged
keilynrp merged 1 commit into
mainfrom
docs/close-377-paging
Sep 29, 2026
Merged

keilynrp merged 1 commit into
mainfrom
docs/close-377-paging

Conversation

@keilynrp

Copy link
Copy Markdown
Collaborator

Closes #377.

Evidence (2026-09-29, operator's phone)

Path Test Result
Pushover channel "Test" in UKIP at 21:36 UTC, phone in Do Not Disturb Rang through DND, repeated, acknowledged
Healthchecks.io dead man's switch Monitor switched off (UKIP_OPS_MONITOR_ENABLED=0, the app kept serving) Paged after 15 min (5 min period + 10 min grace), through DND. Re-enabled: back up within one cycle

The runbook suggested stopping the container. Switching the monitor off sends Healthchecks.io the same silence without taking the API down.

Records updated

OpenSpec

sev1-paging is archived with every task done. Its 5 requirements are published as openspec/specs/sev1-paging.

Checks

  • test_backup_claim_consistency.py: 6 passed
  • Pre-push gates green (docs only)
  • CI green

🤖 Generated with Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aa7e6c8a81

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +28 to +30
- **WHEN** the database check becomes critical
- **THEN** a Pushover channel receives a priority-2 message with `retry` and
`expire` set

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Make database-outage paging independent of the failed database

When PostgreSQL is actually unavailable, this scenario cannot deliver the promised Pushover page: _db_check records a critical result, but the later unguarded database queries in _scheduler_check make run_operational_checks raise; run_once then builds an evaluation_failed report and dispatch_event itself raises while querying AlertChannel from the same unavailable database. The only recorded exercise used the channel's test action, so it does not validate this production path; a real database outage falls back to the dead-man page after its grace period instead of sending this priority-2 message.

Useful? React with 👍 / 👎.

Both paths passed on the operator's phone on 2026-09-29:

- The Pushover channel's test (21:36 UTC) rang through Do Not Disturb,
  repeated, and was acknowledged.
- With the monitor switched off (UKIP_OPS_MONITOR_ENABLED=0; the app kept
  serving), Healthchecks.io paged after 15 minutes, the 5-minute period plus
  10-minute grace, again through Do Not Disturb. Re-enabling the monitor
  brought the check back up within one cycle. The runbook suggested stopping
  the container; switching the monitor off sends Healthchecks.io the same
  silence without taking the API down.

Recorded in the tabletop's corrective action status, whose other rows were
stale too: #375, #376 and #378 closed on 2026-09-24, with the note that
addresses name the client only from 2026-09-29 (#413). Plan §11 gap 2 is
closed and §4 says paging is live; what remains is one person receiving
every page (gap 1) and no paging on access (gap 6). The privacy controls
overview and the Mexico annex no longer list paging as pending.

sev1-paging is archived with all tasks done, and its five requirements are
published as openspec/specs/sev1-paging.

Closes #377.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@keilynrp
keilynrp force-pushed the docs/close-377-paging branch from aa7e6c8 to 42b1cc6 Compare September 29, 2026 23:24
@keilynrp
keilynrp merged commit a32a50d into main Sep 29, 2026
53 checks passed
@keilynrp
keilynrp deleted the docs/close-377-paging branch September 29, 2026 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nothing pages: detection took 6h22m overnight and came from a third party

1 participant