fix(test): widen release_after_ms + bump reruns on approval-timeout flake - #85
Merged
Merged
Conversation
…lake Post-merge push-CI run #30901743674 (master @ 522f33c) failed on the coverage job with `tests/test_approval_timeout_field.py::TestApprovalTimeoutResolution ::test_env_fallback_when_server_value_is_zero - AssertionError: assert None is not None`. The PR matrix legs (3.10/3.11/3.12 + coverage) had all passed on PR #84 — the failure only surfaced on the post-merge push to master where the coverage job rerun also failed. Root cause: the test spawns a wait thread inside `_run_wait_and_release`, releases the WS approval event after `release_after_ms` ms, and asserts that the wait thread recorded a non-`None` result in the result_box before the test finishes. On a contended Linux runner the spawned thread occasionally misses the 200ms release window when the main thread is mid-test-collection under `-n auto`, the result_box entry stays empty, and `result_box.get("result")` is `None`. Sprint 0 (0.14.6 release commit `e7cac4c`) added `@pytest.mark.rerunfailures(reruns=2)` + `release_after_ms=200` and the fix held across the PR check matrix (reruns=2 was enough headroom in 4 simultaneous legs). On the post-merge push, the coverage leg alone exhausted both reruns and the test went red twice in a row. Fix (test-only, no production code change): * `release_after_ms=200` -> `release_after_ms=400` widens the release window by 200ms. Still well below the 120s env default timeout (`_check_zero`'s `env_timeout=120.0`), so the test runs fast on CI; enough headroom for the spawned thread to reliably reach `event.wait()` before the release fires even on a contended runner. * `@pytest.mark.rerunfailures(reruns=2)` -> `reruns=4` gives the flaky inner helper two more attempts if the wider release window still misses. 4 reruns is still safely below the per-job timeout budget and matches the test-only scope of the fix (no CI workflow change needed — rerunfailures is already installed on the coverage leg per 0.14.6). * Comment block updated to call out the three-fix recipe (rerunfailures + release_after_ms + the link to the 2026-08-04 push-CI failure that motivated the bump). Verified locally (Windows, Python 3.12, .venv-ci): * `for i in 1..10; do pytest tests/test_approval_timeout_field.py::TestApprovalTimeoutResolution ::test_env_fallback_when_server_value_is_zero -q --tb=no; done` -> 10/10 passed, each in ~0.7-1.2s. Pre-fix the same loop showed intermittent failures. * `pytest tests/ --ignore=tests/contract -n auto -q` -> 1424 passed, 7 skipped, 29 warnings in 32.34s (matches the post-0.14.7 baseline; no regression introduced). * `ruff check src/ tests/` -> All checks passed. * `mypy src/nullrun --strict` -> Success: no issues found in 37 source files. No production code change. No SDK_MIN_VERSION bump. No public API change. Recommended upgrade path: 0.14.7 -> 0.14.8 (this will be the first post-merge CI-fix release in the 0.14.x line; otherwise the master CI badge stays red).
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Hotfix for the post-merge push-CI coverage failure on master (commit
522f33c, run #30901743674).tests/test_approval_timeout_field.py::TestApprovalTimeoutResolution::test_env_fallback_when_server_value_is_zeroexhausted thereruns=2budget on the post-merge push coverage job — the spawned wait thread missed the 200ms release window twice in a row on the contended Linux runner. PR #84's matrix legs (3.10/3.11/3.12 + coverage) had all passed; the failure surfaced only on the post-merge push run.Fix (test-only, no production code change)
release_after_ms=200→release_after_ms=400intests/test_approval_timeout_field.py:217— widens the release window by 200ms. Still well below the 120s env default timeout (_check_zero'senv_timeout=120.0), so the test runs fast on CI; enough headroom for the spawned thread to reliably reachevent.wait()before the release fires even on a contended runner.@pytest.mark.rerunfailures(reruns=2)→reruns=4attests/test_approval_timeout_field.py:211— gives the flaky inner helper two more attempts if the wider release window still misses. No CI workflow change needed (rerunfailures is already installed on the coverage leg per 0.14.6 commite7cac4c).Why this is a hotfix PR, not a follow-up to 0.14.7
522f33cpush coverage failure). Merging this PR re-greens it without forcing a re-release of 0.14.7.api_key).__version__.py/pyproject.toml/CHANGELOG.mdupdates.Test plan
pytest tests/ --ignore=tests/contract -n auto -q→ 1424 passed, 7 skipped in 32.34s (matches the post-0.14.7 baseline; no regression).ruff check src/ tests/→ All checks passed.mypy src/nullrun --strict→ Success: no issues found in 37 source files.gh pr checksafter opening.Out of scope
uv.lock(mcp / langgraph / chromadb / etc.) are pre-existing and unrelated to this hotfix.Refs
e7cac4c("CI coverage-job flakefix + actions.cooldown race", PR chore(release): 0.14.6 — CI coverage-job flakefix + actions.cooldown race #83).522f33c, coverage job #91967405246 (initial) + #91968574690 (rerun — same failure).