Resolve a cursor the hub declined without waiting on files - #883
Merged
Conversation
The file source it fell back to cannot start before the merged bundle covering that block number exists — twenty minutes on Ethereum — and cannot resolve the cursor when it gets there. Undo to its LIB right away instead.
Contributor
Author
sduchesneau
marked this pull request as ready for review
August 17, 2026 19:12
maoueh
reviewed
Aug 17, 2026
| return cursor.LIB, headBlock, nil | ||
| } | ||
|
|
||
| src = bstream.NewFileSourceFromCursor(mergedBlocksStore, forkedBlocksStore, cursor, jctBlkGetter, zap.NewNop(), fileSourceOptions...) |
Contributor
There was a problem hiding this comment.
So are we double doing the CheckCursorResolvable now? Isn't NewFileSourceFromCursor doing the same?
Contributor
Author
There was a problem hiding this comment.
The FileSourceFromCursor just determines that the file is in the future, so it waits for the merged block to appear...
Also, it cannot return an error, only a filesource or nil.
So that's 20 minutes on eth-mainnet !
CheckCursorResolvable checks if we are in the HEAD section, if so: if a forked-block contains it. If we are older than the LIB, we return nil (not our concern) so it goes back to the fileSourceFromCursor.
This prevents waiting for merged-blocks to be produced.
maoueh
approved these changes
Aug 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
NewCursorResolverasks the hub first and falls back tobstream.NewFileSourceFromCursorwhen it returns nil. The hub returns nil for two different reasons, and one of them is not "this cursor is historical": it also declines a cursor whose block number it covers but whose ID it has never seen — a corrupted or forged cursor, or one carried over from a chain this process never served.That file source cannot start before the merged-blocks bundle covering the cursor's block number is written. Near chain head it is not, so the request sits silent for a whole bundle — measured at ~16 minutes against
mainnet.eth.streamingfast.iowith a cursor whose block hash had its last 4 hex rewritten — and the files cannot resolve it when they arrive either.Reproduced locally on
firehose-core/devel/standardwith the merger stopped, so the bundle never lands:developreturns nothing at all in 90s, no undo, no error.Change
Before the file-source fallback, ask whether anything can still produce that block (
bstream.CheckCursorResolvable, added in bstream#63):[LowestBlockNum, HeadNum], so a cursor block in that range whose ID it does not know is on no chain it ever saw;cursor.LIB, the existing "resolver failed" outcome, whichresolveStartBlockNumturns into aBlockUndoSignalto LIB plus a start block ofLIB+1.The client gets the undo immediately instead of after the merger catches up — the same outcome, minus the twenty minutes.
A cursor above the hub's head is deliberately not that case: the block exists, this instance has not reached it, which is what a client reconnecting to an instance a few blocks behind the one that served it looks like. bstream waits it out briefly and reports
ErrCursorAboveHead; this PR propagates that asconnect.CodeUnavailable(retryable) instead of the usualCodeInvalidArgument, so a lagging tier1 neither kills a sink nor reverts one that is ahead of it.Measured, local stack with the merger stopped
developUNDO … number:2910then blocks, immediatecode = Unavailable, which the sink logs as a retryable errorVerified against the real thing too: the same forged-cursor scenario on
mainnet.eth.streamingfast.iois what surfaced this.