Do not consume task identity on add_job conflict - #623
Open
Steve0x2a wants to merge 1 commit into
Open
Conversation
Insert missing task/queue identifiers only so nextval is not called when the name already exists.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
insert ... on conflict do nothinginto_private_tasks/_private_job_queuesstill callsnextval()before the unique check, so every enqueue burns an integer identity. After ~2.1bn cumulative enqueues the sequence hits 2147483647 and bothadd_joband worker startup fail, even when the identifier table has only a handful of rows.This changes those inserts (in
add_jobs,add_jobunsafe_dedupe, andgetTaskDetails) to insert only missing identifiers (WHERE NOT EXISTS), so already-registered names do not consume identity.ON CONFLICT DO NOTHINGis kept for the concurrent-insert race.Reported by @perebusquets.
Fixes #619.