Logfill bounded tranlog stream - #6171
Open
markhannum wants to merge 2 commits into
Open
Conversation
roborivers
suggested changes
Aug 28, 2026
roborivers
left a comment
There was a problem hiding this comment.
Cbuild submission: Error ⚠.
Regression testing: Success ✓.
The first 10 failing tests are:
logfill [db unavailable at finish] **quarantined**
sc_resume_logicalsc_generated **quarantined**
sp_snapshot_generated
comdb2sys_queueodh_generated
consumer_non_atomic_default_consumer_generated **quarantined**
sc_downgrade [timeout] **quarantined**
An unbounded stream never ends, so logfill abandoned it mid-result-set and cdb2api dropped the socket rather than donating it back to sockpool. Also stop tranlogNext from blocking once it has returned maxLsn. Signed-off-by: Mark Hannum <[email protected]>
Drops every Nth rep message on one replicant to force gaps, then counts log-requests against master disconnects: an abandoned result set forces a disconnect, so the two track one-for-one when the stream is unbounded. Signed-off-by: Mark Hannum <[email protected]>
markhannum
force-pushed
the
logfill-bounded-tranlog-stream
branch
from
August 28, 2026 14:51
cf8657c to
2873bd0
Compare
roborivers
suggested changes
Aug 28, 2026
roborivers
left a comment
There was a problem hiding this comment.
Cbuild submission: Error ⚠.
Regression testing: Success ✓.
The first 10 failing tests are:
logfill [db unavailable at finish] **quarantined**
ssl_san
consumer_non_atomic_default_consumer_generated **quarantined**
ssl_set_cmd
ssl_prefer
ssl_dbname
sc_downgrade [timeout] **quarantined**
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A replicant running
sql_logfilldestroys one connection to the master per gap-fill, and each of those is a sockpool miss. Observed in production at roughly 890 log-requests/sec across seven replicants, sustained for hours.Cause
request_logs_from_master()asks for an unbounded stream (NULLstop LSN,TRANLOG_FLAGS_BLOCK), so it never ends on its own. logfill consumes only as far as the gap and abandons the rest — in production the master averaged 636 rows per query against a gap of a couple of records.newsql_disconnect()will not donate a handle whose last response was notLAST_ROW, andcdb2_discard_unread_socket_datadefaults to off, so the socket is closed rather than pooled and the next request misses.This happens on both sides of
0ed0d3f3b: with it, logfill hangs up and reconnects explicitly; without it,consume_previous_query()drains at most 10 rows and then disconnects from insidecdb2_run_statement.Fix
Pass
gap_lsnas the stop LSN.tranlogEof()already terminates on it, so the result set ends withLAST_ROW, the existingconsumedpath skips the disconnect, and the connection persists across gap-fills.A second hunk stops
tranlogNext()from entering the blocking poll once it has returnedmaxLsn— that loop checks onlyblockLsnandtimeout, so otherwise the bound would trade connection churn for a stall.Test
tests/sql_logfill_socket_reuse.testdrops every Nth rep message on one replicant to force out-of-order arrivals, then counts log-requests against master disconnects. A gap is an out-of-order arrival rather than simply being behind —gap_lsnis repdb'swaiting_lsn— so dropping messages is what reproduces the production condition. Unbounded, disconnects track requests one-for-one; the test allows a tenth. Verified 8 consecutive passes with the fix, and confirmed failing without it.