Repository navigation
Keep the cron on the acceptance deployment - #48
Merged
Merged
Conversation
Run 14 timed out waiting for a watermarked rendition even after the sweeps were wired into the scheduled handler, because this generator deletes the cron that calls it. It was deleted to stop a throwaway Worker firing every minute until the delete landed. That was the wrong trade: the scheduled handler is what queues a watermark, backfills a still ladder and buries an abandoned job, so removing it made the acceptance run test a deployment that could never do any of those things. The run then failed for a reason a real deployment would not have had, and the sweep fix could not have helped. A run lives about fifteen minutes, so this costs about fifteen invocations.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Run 14 timed out waiting for a watermarked rendition even after #46 wired the sweeps into the scheduled handler. The reason is in this generator:
I removed the cron so a throwaway Worker would not fire every minute until the delete landed. But the scheduled handler is what runs the sweeps, and the sweeps are what queue a watermark, backfill a still ladder and bury an abandoned job.
So the acceptance run was deploying a Worker that could never do any of those things, and then failing because it didn't. My own optimisation made the test unrepresentative of production, and I then spent a run testing against it.
A run lives about fifteen minutes, so keeping the cron costs roughly fifteen invocations of a handler that is deleted afterwards.
Worth noting the sweep work in #46 is still correct and still needed — it just could not have been observed through this harness.