Skip to content

fix(eth/tracers): fix nonce too high and null entries in block traces - #2581

Open
gzliudan wants to merge 1 commit into
XinFinOrg:dev-upgradefrom
gzliudan:fix-nonce-too-high
Open

gzliudan wants to merge 1 commit into
XinFinOrg:dev-upgradefrom
gzliudan:fix-nonce-too-high

Conversation

@gzliudan

Copy link
Copy Markdown
Collaborator

Proposed changes

debug_traceBlockByNumber, debug_traceBlockByHash and debug_traceBlock dropped every transaction sent to the XDCX system addresses 0x91/0x92/0x93/0x94, with an unconditional tx.IsSkipNonceTransaction() check that does not consult the fork at all.

Outside the XDCX receiver fork window (mainnet TIPXDCXBlock 38,383,838TIPXDCXReceiverDisableBlock 80,370,900) block processing does execute those transactions and does bump the sender nonce, so dropping them lost exactly the nonce increment the following transaction of the same sender relies on, and the whole call failed with nonce too high. Inside the window the skip kept the state right, but left a null hole in the result array where the trace of a transaction should be.

Two loops skipped them: the transaction loop of traceBlock, and the state feeder of traceBlockParallel, which only tracers that evaluate JS code reach. The feeder had a second divergence: it advanced the state the workers trace against with core.ApplyMessage, which knows nothing about the routing block processing applies, so while the fork was active it bumped a nonce the chain leaves alone.

Both loops now replay through core.ApplyTransactionWithEVM, the entry point the serial tracer already uses, which routes those addresses the way block processing does, so every transaction of the block gets one entry and the pre-state handed to the next trace stays in step with execution.

Symptoms

  • nonce too highdebug_traceBlockByNumber / ByHash / traceBlock on a block outside the receiver fork window that contains a transaction to a system address followed by another transaction of the same sender. The skipped transaction is the one that bumped the nonce, so the follower is replayed one nonce short. Mainnet block 37,849,457: tracing failed: nonce too high: address xdcE3bC38418f89C386d1093aee89d606564012e329, tx: 1547093 state: 1547092; the whole call fails, no trace at all.
  • The same failure on the parallel path: a tracer given as an inline JS expression fails with nonce too high on the same block, because DefaultDirectory.IsJS answers true for a name that is not registered, so such a tracer sends traceBlock through traceBlockParallel and through its state feeder.
  • One null entry instead of a trace for those transactions in debug_traceBlockByNumber / ByHash / traceBlock, whenever the fork is active (mainnet block 39,083,312).

Out of scope

debug_intermediateRoots (a root missing for those transactions, and a last root that diverges from the block state root when the block charges a fee), flatCallTracer (invalid number of calls on any block containing a sign transaction) and debug_traceTransaction (nonce too low on a transaction that follows a system-address transaction while the fork is active) are separate defects with the same root cause. They are not touched here; this PR is limited to the two loops above.

Upstream

No upstream fix to port: geth has no non-EVM transaction concept, so neither the unconditional skip nor the trailing ApplyMessage have a geth counterpart.

Tests

  • eth/tracers/api_test.goTestTraceBlockSkipNonceTransactions: drives TraceBlockByNumber over both receiver fork settings, with the default struct logger and with a named tracer, and asserts the block gets one result per transaction with no null entry.
  • eth/tracers/api_test.goTestTraceBlockParallelSkipNonceTransactions: the same over traceBlockParallel, reached with a registered tracer that reports isJS = true. Its two sub-cases pin the two directions of the bug, and neither half of the fix passes both: a feeder that keeps the skip fails the outside-the-window case with nonce too high, one that drops the skip but keeps ApplyMessage fails the inside-the-window case with nonce too low.
  • The state tracer is registered in init() so that parallel tests cannot race on the tracer directory, and the probe tracer that reports isJS reaches the parallel path with the JS evaluator not linked into the test binary.

End-to-end verification

The binary built from this commit was run against a mainnet archive node, on the same data directory and the same node as the baseline binary (dev-upgrade @ cdce8fc5c), and the tracing RPCs were compared on the same blocks.

Mainnet block 37,849,457 (0x2418971, 204 transactions, outside the receiver fork window; tx 0 goes to 0x92, tx 95 to 0x90 from the same sender with nonce +1):

probe baseline this branch
debug_traceBlockByNumber (callTracer) tracing failed: nonce too high: address xdcE3bC38…, tx: 1547093 state: 1547092 204 entries, none null
debug_traceBlockByNumber (an inline JS tracer, the parallel path) nonce too high: address xdcE3bC38…, tx: 1547093 state: 1547092 204 entries, none null

Mainnet block 39,083,312 (0x2545d30, 14 transactions, inside the window; tx 0 goes to 0x92, tx 5 to 0x90 from the same sender with the same nonce):

probe baseline this branch
debug_traceBlockByNumber (callTracer) 14 entries, one of them null 14 entries, none null

Manual test plan: start a node on an archive data directory with --rpcapi debug and call debug_traceBlockByNumber on 0x2418971 and 0x2545d30, with callTracer and with an inline JS tracer; the baseline binary fails both blocks and this branch passes them.

Regression: the node imported mainnet segments normally, with no bad block, no panic and no error attributable to the change.

Types of changes

  • fix: A bug fix
  • test: Adding missing tests or correcting existing tests
  • build / ci / chore / docs / feat / perf / refactor / revert / style

Impacted Components

  • Geth
  • Consensus
  • Account
  • Network
  • Smart Contract
  • External components
  • Not sure

Checklist

  • This PR has sufficient test coverage (unit/integration test)
  • Provide an end-to-end test plan in the PR description on how to manually test it on the devnet/testnet (see "End-to-end verification")
  • Tested the backwards compatibility — the change only affects replays; block processing keeps the same conditions and nonce handling, and no API, RPC method or state format changes
  • Tested on a private network from the genesis block and monitored the chain operating correctly for multiple epochs — not done for this branch. It was run against a live mainnet archive node instead (segments imported, no bad block, no panic, no new error), and the tracing RPCs were compared block by block against the baseline binary.
  • Tested with XDC nodes running this version co-exist with those running the previous version — the branch ran on the live network next to the rest of the network without being dropped
  • Relevant documentation has been updated as part of this PR

Relation to other work

#2579 contains this commit as its first one (same sha), followed by the fixes for the defects listed under "Out of scope" and a shared core.ApplyTransactionForReplay entry point. Merging this PR first keeps #2579 a trivial rebase: its first commit is already applied and is dropped by patch id. #2578 touches eth/state_accessor.go only and shares no file with this PR.

@coderabbitai

coderabbitai Bot commented Sep 18, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: a8083b57-6073-45d7-bf35-ae8a0b8329f5

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The implementation and regression coverage address both routing cases; only minor comment inaccuracies remain.

Pull request overview

Fixes block tracing so XDCX system-address transactions follow block-processing routing and remain present in trace results.

Changes:

  • Removes unconditional skip-nonce filtering.
  • Replays parallel feeder transactions through ApplyTransactionWithEVM.
  • Adds fork-active and fork-inactive regression tests.
File summaries
File Description
eth/tracers/api.go Aligns serial and parallel tracing with transaction execution.
eth/tracers/api_test.go Covers nonce handling and complete trace results.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread eth/tracers/api.go Outdated
Comment thread eth/tracers/api_test.go Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused fix aligns tracing with block execution and includes targeted regression coverage.

Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The call tracer can return a synthetic trace for transactions that execute through the EVM outside the fork window.

Get a fresh assessment by requesting another Copilot review.

Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread eth/tracers/api.go
Comment thread eth/tracers/api_test.go
debug_traceBlockByNumber, debug_traceBlockByHash and debug_traceBlock dropped
every transaction sent to the XDCX system addresses 0x91/0x92/0x93/0x94, with
an unconditional tx.IsSkipNonceTransaction() check that does not consult the
fork at all. Outside the receiver fork window block processing does execute
those transactions and does bump the sender nonce, so dropping them lost
exactly the nonce increment the following transaction relies on and the whole
call failed with "nonce too high" (mainnet block 37849457). While the fork is
active the skip kept the state right, but left a null hole in the result array
where the trace of a transaction should be.

Two loops skipped them: the transaction loop of traceBlock, and the state feeder
of traceBlockParallel, which only tracers that evaluate JS code reach. The
feeder had a second divergence: it advanced the state the workers trace against
with core.ApplyMessage, which knows nothing about the routing block processing
applies, so while the fork is active it bumped a nonce the chain leaves alone.

Trace them instead. Both loops now replay through core.ApplyTransactionWithEVM,
which routes those addresses the way block processing does, so every
transaction of the block gets one entry and the pre-state handed to the next
trace stays in step with execution.

Add TestTraceBlockSkipNonceTransactions and
TestTraceBlockParallelSkipNonceTransactions, which drive TraceBlockByNumber over
both receiver fork settings on the serial path and on the parallel one, with the
default struct logger and with a named tracer, together with the two fixtures
the skip-nonce tests share, skipNonceForkCases and newSkipNonceBackend. The
state tracer is registered in init() so that parallel tests cannot race on the
tracer directory, and the probe tracer that reports isJS reaches the parallel
path with the JS evaluator not linked into the test binary.

No corresponding fix exists upstream: geth has no non-EVM transaction concept.

Refs: gzliudan/XDPoSChain#371, gzliudan/XDPoSChain#260

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The implementation aligns both tracing paths with block execution and the tests cover both nonce-routing branches and result completeness.

Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants