Let a retry policy decide will_be_retried in the runner - #345
Open
Enceradeira wants to merge 2 commits into
Open
Let a retry policy decide will_be_retried in the runner#345Enceradeira wants to merge 2 commits into
Enceradeira wants to merge 2 commits into
Conversation
5 tasks
Enceradeira
force-pushed
the
fix/cucumber-ruby-1905-will-be-retried
branch
2 times, most recently
from
September 5, 2026 14:51
19900a2 to
d98745a
Compare
The runner emits the TestCaseFinished envelope after broadcasting the test_case_finished event. The retry filter in cucumber-ruby re-runs the test case from inside that event, so the envelope of the first attempt is emitted after the whole retry and carries the test_case_started id of the retry instead of its own. The runner also derives will_be_retried from a maximum number of attempts, which it cannot know: whether a test case runs again is decided by whoever runs it again, and the retry filter stops retrying once --retry-total is exhausted. The runner has to ask a retry policy. These specs fail until the runner emits the envelope before the event and asks a retry policy. Related to cucumber/cucumber-ruby#1905.
Enceradeira
force-pushed
the
fix/cucumber-ruby-1905-will-be-retried
branch
from
September 5, 2026 14:55
d98745a to
3419704
Compare
The runner computed the will_be_retried field of the TestCaseFinished envelope from a maximum number of attempts. It cannot know that on its own: whether a test case runs again is decided by whoever runs it again, which in cucumber-ruby is the retry filter honouring both --retry and --retry-total. The runner now asks a retry policy instead. The envelope is also emitted before the test_case_finished event. The retry filter re-runs the test case from inside that event, so the envelope of the first attempt used to be emitted after the retry, with the test_case_started id of the retry. Related to cucumber/cucumber-ruby#1905.
Enceradeira
force-pushed
the
fix/cucumber-ruby-1905-will-be-retried
branch
from
September 5, 2026 14:57
3419704 to
52656e0
Compare
This was referenced Sep 5, 2026
Open
Enceradeira
marked this pull request as ready for review
September 5, 2026 15:51
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Alternative to #344, fixing cucumber/cucumber-ruby#1905 together with cucumber/cucumber-ruby#1910.
Since 19.0.0 the
Runneremits theTestCaseFinishedenvelope and decides itswill_be_retriedfield from a maximum number of attempts. The runner cannot know that on its own: whether a test case runs again is decided by whoever runs it again, which in cucumber-ruby is the retry filter honouring both--retryand--retry-total. #344 copies the--retry-totalcounter into the runner, which makes the reported symptom go away but keeps the retry decision in two places, and leaves two further defects in the message stream:--retry 1the flag is nevertrue, so a retried attempt is reported as a final result and the HTML report shows the scenario twice. With--retry 2the second attempt is reported as final although a third one follows.test_case_finishedevent before emitting theTestCaseFinishedenvelope. The retry filter re-runs the test case from inside that event, so the envelope of the first attempt is emitted after the retry and carries thetest_case_started_idof the retry. Onmain, a retried scenario produces twoTestCaseFinishedenvelopes for attempt 2 and none for attempt 1.The compatibility kit does not catch any of this because it only compares message keys, not values.
This PR:
max_attemptsconstructor argument with aretry_policy, an object answeringwill_be_retried?(test_case, result). With no policy, no test case is reported as retried.TestCaseFinishedenvelope before broadcasting thetest_case_finishedevent, so a retry started from that event no longer reorders the stream.Type of change
Cucumber::Core::Test::Runner), seeupgrading_notes/20.0.0.mdChecklist:
bundle exec rubocopreports no offenses