Skip to content

fix(roadmap): fail a status read in its own row - #4143

Open
dan-petty wants to merge 6 commits into
mainfrom
fix/4052-status-read-own-row
Open

dan-petty wants to merge 6 commits into
mainfrom
fix/4052-status-read-own-row

Conversation

@dan-petty

Copy link
Copy Markdown
Owner

The status row metrics read the project metrics, the GitHub status page and the Cloudflare status page, and hid each failure in an except Exception that only logged. A failed read counted as success, so the job counter, the failure record and the alert RoadmapServiceJobsFailing never saw it. The row now raises. The two status pages have their own rows, github_status and cloudflare_status, so one failed read fails only its own row. No setting changes.

Closes #4052

Change: openspec/changes/archive/2026-11-08-4052-a-status-read-that-fails-fails/

@dan-petty dan-petty added type/bug Defect, regression, or incorrect runtime behavior scope/roadmap The roadmap jobs: intake, refine, stage, implement and the Service that runs them labels Nov 8, 2026
@dan-petty

Copy link
Copy Markdown
Owner Author

Test record for #4052.

Fail-to-pass run (uv run devops ci fail-to-pass --base origin/main, on the test commit): 18 tests failed on the base, 100 passed, 0 errors, in 9 files under tests/roadmap/. The 18 failures are the new tests of tests/roadmap/test_roadmap_status_rows.py (row_names, source_failed, ledger_per_row) and the row-list tests that now name six rows. Output: .data/agent/state/4052-fail-to-pass.txt.

Full devops ci on the final code, under the shared lock:

  • All rows passed except swallowed failures and the test time budget.
  • swallowed failures: a ratchet gain (roadmap/run.py went from 4 blind excepts to 1). I lowered the ceiling in commit test(gate): lower the swallowed failure ceiling of roadmap run. A rerun of that row passed.
  • test: pytest and coverage passed. The row took 14m 3s, over the 5 minute budget. This is the known fix(ci): the pytest step of devops ci exceeds the 5-minute budget #3593; the slowest test is in tests/prometheus/, not in this change.
  • After a rebase onto origin/main (only k8s/README.md and docs changed there): devops docs generate --check ok; status rows, ConfigMap key-walk and rollout-rule tests pass (13 tests).

Rollout: no setting is removed, renamed or tightened. The example and live ConfigMap name no status row, so no ConfigMap change. The new rows github_status and cloudflare_status reuse the metrics interval. An existing schedule.json without them makes them due once on the first run. A ledger entry for metrics stays valid. Nothing was applied to the cluster.

@dan-petty
dan-petty marked this pull request as ready for review November 8, 2026 06:08

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scope/roadmap The roadmap jobs: intake, refine, stage, implement and the Service that runs them type/bug Defect, regression, or incorrect runtime behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(roadmap): a status read that fails fails its own row

1 participant