You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Flaky unit test: TestReaderReconnector.test_reconnect_on_repeatable_error (timeout on 3.12) #850
ydb/_topic_reader/topic_reader_asyncio_test.py::TestReaderReconnector::test_reconnect_on_repeatable_error fails intermittently in CI with a timeout. Recurring flake — has been seen on several unrelated PRs.
Failure
ydb._topic_common.test_helpers.WaitConditionError: Bad condition in test
ydb/_topic_common/test_helpers.py:67: WaitConditionError
The test drives ReaderReconnector through one repeatable error (Overloaded) and then a healthy stream, and waits via:
wait_for_fast → wait_condition budgets 1s wall / 1000 loop iterations (test_helpers.py:46-67). When reconnector.wait_message() does not resolve within that budget the helper raises WaitConditionError. So this is a timing-sensitive timeout, not an assertion about reconnect behavior.
A teardown side-effect also shows up after the failure (likely secondary, from the aborted coroutine):
Reproduces locally only occasionally; ~7s suite, fails at ~54% progress.
Hypothesis / directions to investigate
The reconnect path (error stream → recreate → first message) occasionally needs more than the 1s / 1000-iteration budget under CI load, especially on 3.12's event-loop scheduling. Bumping the budget would only mask it.
Worth checking whether ReaderReconnector does extra await asyncio.sleep(0) hops on the reconnect path on 3.12, or whether a backoff/retry delay leaks into the loop and eats the budget.
Confirm the Event loop is closed teardown warning is purely secondary (GC of the parked wait_forever/Queue.get coroutine) and not contributing to the hang.
Workaround for now
Re-run the job; failure is intermittent and not a regression from the triggering PR.
I investigated this failure and prepared a local draft. The retry path for Overloaded uses a randomized slow backoff of 0.5–1s, while wait_for_fast times out after 1s. I propose setting the slow backoff to zero only in this test, synchronizing the mocked stream callbacks, and closing the reconnector in finally. This keeps production retry behavior unchanged. Would this be an acceptable approach?
I investigated this failure and prepared a local draft. The retry path for Overloaded uses a randomized slow backoff of 0.5–1s, while wait_for_fast times out after 1s. I propose setting the slow backoff to zero only in this test, synchronizing the mocked stream callbacks, and closing the reconnector in finally. This keeps production retry behavior unchanged. Would this be an acceptable approach?
Activity
duckduckduck1 commentedon Sep 24, 2026
@vgvoleg, Hi!
I investigated this failure and prepared a local draft. The retry path for
Overloadeduses a randomized slow backoff of 0.5–1s, whilewait_for_fast timesout after 1s. I propose setting the slow backoff to zero only in this test, synchronizing the mocked stream callbacks, and closing the reconnector infinally. This keeps production retry behavior unchanged. Would this be an acceptable approach?vgvoleg commentedon Oct 6, 2026
Hi! of course, thanks for contribution!
duckduckduck1 commentedon Oct 23, 2026
@vgvoleg
Thanks for confirming!
I’ve opened PR #924 with the proposed test-only changes: #924