Skip to content

fix(core,eth,miner): fix traces and intermediate roots of the TIPSigning activation block - #2579

Closed
gzliudan wants to merge 1 commit into
XinFinOrg:dev-upgradefrom
gzliudan:fix-trace-replay
Closed

gzliudan wants to merge 1 commit into
XinFinOrg:dev-upgradefrom
gzliudan:fix-trace-replay

Conversation

@gzliudan

@gzliudan gzliudan commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Proposed changes

Block processing removes the legacy block signers account (common.BlockSignersBinary) before the first transaction of the block that activates TIPSigning (mainnet TIPSigningBlock 3,000,000), the same way it applies the DAO hard fork at the DAO fork block. None of the replays ran that removal: stateAtTransaction (the replay behind debug_traceTransaction and debug_traceCall), IntermediateRoots (behind debug_intermediateRoots) and the per-block state builder traceChain uses all rebuild the block from its parent state, so they keep an account canonical execution had already deleted, and the traces and every intermediate root of the activation block describe a state the chain never had.

The chain makers had the same gap, which is what makes it visible in tests rather than only on the live networks: GenerateChain mirrors the block level mutations of the DAO fork and of Prague, so a generated chain that activates TIPSigning produces a block whose root does not match what block processing computes, and such a block cannot be imported at all.

Symptoms

  • Every trace and every intermediate root of the TIPSigning activation block (mainnet block 3,000,000) is computed on a state that still holds the legacy block signers account block processing had already removed before the first transaction of the block, so the first debug_intermediateRoots root differs.
  • A generated chain whose blocks activate TIPSigning cannot be imported: GenerateChain produces a state root block processing does not reproduce (invalid merkle root).

Fix

The removal moves into core.ApplyTIPSigningHardFork, which Process, ProcessBlockNoValidator and the miner call instead of each keeping its own copy of the condition, and which the three replay paths above and GenerateChain call as well. The condition is unchanged — it fires on the activation block only — so block processing and the miner build the same blocks as before.

Upstream

No upstream fix to port: geth has no TIPSigning fork, so the block-level removal the replays missed has no counterpart. It has no non-EVM transaction concept either, which is why the ApplyMessage replays the other two PRs of this series replace have none.

Tests

  • eth/tracers/api_tipsigning_activation_test.goTestIntermediateRootsMatchesBlockProcessingAtTIPSigningActivation: traces a block that activates TIPSigning over a parent state that holds the legacy account, pins that premise from both sides (the account is in the parent state, and it is gone from the state the imported block carries) and compares the intermediate roots against a block processing replay pinned to block.Root(); without the call the very first root already differs. The file carries its own mirror of the block processing replay, so it does not build on the helper the intermediate-roots PR adds.

End-to-end verification

The branch binary was run against archive nodes of both live networks, on the same data directory and the same node as the baseline binary (dev-upgrade @ cdce8fc5c): on both networks this branch imported testnet and mainnet segments normally, with no bad block, no panic and no error attributable to the change.

The activation block itself was not probed on the live networks, because it is far outside the segments the archive nodes re-import. Its coverage is the unit test above, which pins its own premise from both sides and checks the comparison path against block.Root().

Manual test plan: start a node on mainnet archive data with --rpcapi debug and call debug_intermediateRoots and debug_traceTransaction on mainnet block 3,000,000; the baseline binary replays that block without the removal, so its first intermediate root already differs from the one block processing produces.

Types of changes

  • fix: A bug fix
  • refactor: A code change that neither fixes a bug nor adds a feature
  • test: Adding missing tests or correcting existing tests
  • build / ci / chore / docs / feat / perf / revert / style

Impacted Components

  • Geth
  • Not sure (the changes are the debug JSON-RPC surface, plus the TIPSigning removal it now shares with the miner and the chain makers)
  • Consensus
  • Account
  • Network
  • Smart Contract
  • External components

Checklist

  • This PR has sufficient test coverage (unit/integration test)
  • Provide an end-to-end test plan in the PR description on how to manually test it on the devnet/testnet (see "End-to-end verification")
  • Tested the backwards compatibility — the change only affects replays and the chain makers; block processing and the miner keep the same condition, finalisation and nonce handling, and no API, RPC method or state format changes
  • Tested on a private network from the genesis block and monitored the chain operating correctly for multiple epochs — not done for this branch. It was run against live testnet and mainnet archive nodes instead (segments imported, no bad block, no panic, no new error).
  • Tested with XDC nodes running this version co-exist with those running the previous version — the branch ran on the live networks next to the rest of the network without being dropped
  • Relevant documentation has been updated as part of this PR

Relation to other work

This is the third of three PRs that split a single debug tracing series, so every defect is reviewable on its own.

#2587 carries the shared block processing routing and the replay entry point, and rewrites the stateAtTransaction replay — the nonce too low of debug_traceTransaction. #2588 fixes the missing intermediate roots of a block. Neither is a prerequisite for this one: the removal introduced here is independent of the routing they share, and the call sites it adds are all in code the base already has.

This PR carries its own mirror of the block processing replay in its test, because the shared helper comes with #2588. When the other two land, that copy is dropped in favour of the shared helper, which is the only change this branch needs on rebase.

#2584 carries the two eth/tracers fixes this series used to contain — callTracer shadowing the real top-level frame of a system-address transaction, and flatCallTracer failing the whole call with invalid number of calls on a block that carries a non-EVM transaction.

#2581 fixes the unconditional skip in traceBlock and in the state feeder of the JS tracer path — the nonce too high and the null hole in debug_traceBlock*.

#2578 fixes the give-up paths of the same stateAtTransaction function #2587 rewrites.

@coderabbitai

coderabbitai Bot commented Sep 18, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 6b22a890-7e89-4b19-98e9-83b9aa4fad8f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gzliudan gzliudan changed the title fix(eth,core): make debug tracing match block processing fix(core,eth): fix nonce errors and wrong traces, roots and frames in debug tracing Sep 18, 2026
@gzliudan gzliudan changed the title fix(core,eth): fix nonce errors and wrong traces, roots and frames in debug tracing fix(core,eth): fix nonce too high/low and wrong traces, roots and frames in debug tracing Sep 18, 2026
@gzliudan
gzliudan requested a balanced review from Copilot September 18, 2026 03:04

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Replay still diverges from canonical processing for synthetic log indexes and sequential TRC21 fee-capacity accounting.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Fixes debug tracing replay so transaction routing and trace frames match canonical block processing.

Changes:

  • Adds a shared routing-aware replay path.
  • Corrects synthetic versus real call frames.
  • Adds regression coverage for nonce, root, and tracer behavior.
File summaries
File Description
eth/tracers/native/call.go Selects real or synthetic call frames correctly.
eth/tracers/native/call_test.go Tests real-frame preservation.
eth/tracers/native/call_flat.go Supports synthetic non-EVM frames.
eth/tracers/native/call_flat_test.go Tests flat tracer frame handling.
eth/tracers/internal/tracetest/non_evm_trace_test.go Tests routing through production processing.
eth/tracers/api.go Replays block state through shared routing.
eth/tracers/api_test.go Adds block, transaction, and root regressions.
eth/state_accessor.go Uses routing-aware transaction replay.
eth/state_accessor_test.go Tests nonce preservation during replay.
core/state_processor.go Centralizes routing and adds replay processing.
Review details

Suppressed comments (2)

eth/tracers/internal/tracetest/non_evm_trace_test.go:74

  • The test only sends to TradingStateAddrBinary and toggles the receiver fork, so it never exercises routeSign/ApplySignTransaction. Add a case using BlockSignersBinary with TIPSigningBlock active; this is especially important because the PR's reported flatCallTracer failure involved a sign transaction and the new replay branch has distinct nonce and log behavior.
	key, _ := crypto.GenerateKey()
	from := crypto.PubkeyToAddress(key.PublicKey)
	to := common.TradingStateAddrBinary
	blockNumber := common.Big1

eth/state_accessor.go:266

  • These replay-error returns discard the release callback obtained from StateAtBlock(..., readOnly=true), so repeated failed traces can retain live trie references. This leak predates this replay change and is already addressed by the linked #2578; ensure that dependency lands before this branch is merged, or include the same deferred release guard here.
		if err := core.ApplyTransactionForReplay(msg, new(core.GasPool).AddGas(tx.Gas()), block.Number(), tx, evm, balance); err != nil {
			// An EVM this replay cannot use is a problem of this caller, not of the
			// transaction: report it as it is instead of blaming the transaction.
			if errors.Is(err, core.ErrReplayTracingEVM) || errors.Is(err, core.ErrReplayStateType) {
				return nil, vm.BlockContext{}, nil, nil, err
			}
			return nil, vm.BlockContext{}, nil, nil, fmt.Errorf("transaction %#x failed: %v", tx.Hash(), err)
  • Files reviewed: 10/10 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread core/state_processor.go Outdated
Comment thread core/state_processor.go Outdated
@gzliudan
gzliudan force-pushed the fix-trace-replay branch 3 times, most recently from eae1995 to 96fd012 Compare September 18, 2026 03:55
@gzliudan gzliudan changed the title fix(core,eth): fix nonce too high/low and wrong traces, roots and frames in debug tracing fix(core,eth): fix nonce too low and wrong traces, roots and frames in debug tracing Sep 18, 2026
@gzliudan gzliudan changed the title fix(core,eth): fix nonce too low and wrong traces, roots and frames in debug tracing fix(core,eth): fix nonce too low and wrong intermediate roots in debug tracing Sep 18, 2026
@gzliudan
gzliudan requested a balanced review from Copilot September 18, 2026 12:13

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Pre-Byzantium replay redundantly hashes trie state for every transaction, adding avoidable cost to historical tracing.

Get a fresh assessment by requesting another Copilot review.

Review details
  • Files reviewed: 6/6 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread core/state_processor.go Outdated
Comment thread eth/tracers/api_test.go Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Shared state-transition routing and historical fork behavior require final human validation.

Review details

Suppressed comments (1)

eth/tracers/api_test.go:1467

  • This helper does not pin its returned last root to block.Root(): roots is populated before consensus finalization, and ethash.Finalize then credits the block reward before final.Root() is checked. The comment currently describes a guarantee the helper deliberately does not provide; clarify that only the separately finalized state is checked against the canonical root.
// replayIntermediateRoots rebuilds the per-transaction state roots of a block with the
// block processing entry point, giving tests an independent expectation for the tracer
// result. It also closes the block the way chain generation does, so the last root is
// pinned to the block root on chain rather than to another replay path.
  • Files reviewed: 6/6 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread eth/tracers/api_test.go Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Core transaction routing and historical state reconstruction changes require final human validation.

Review details
  • Files reviewed: 6/6 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread core/state_processor_test.go Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Core replay-routing changes affect consensus-adjacent state transitions, and trie-reference cleanup remains unresolved.

Review details

Suppressed comments (1)

eth/tracers/api_test.go:1544

  • The only IntermediateRoots regression fixture forces receiver routing active. It covers the missing-root case, but not the removed skip's outside-window failure, where the system-address transaction must execute through the EVM and increment the nonce before a nonce+1 follower. Please parameterize this test with an inactive/disabled-fork case and assert both roots there as well.
	config := *params.TestChainConfig // TIPXDCXBlock is 0: the receiver fork is active
	config.TIPXDCXReceiverDisableBlock = nil
  • Files reviewed: 6/6 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread eth/state_accessor.go Outdated
Comment thread core/state_processor_test.go Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

It changes shared state-processing paths and still has unresolved compilation and activation-block replay defects.

Review effort: Balanced
Findings: 1 High severity · 1 Medium severity

Open (2)
Resolved since last review (2)

Comment thread core/state_processor.go
Comment thread eth/state_accessor.go Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Fork-sensitive state processing needs human validation, and tracing test backends still omit TIPSigning setup.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Resolved since last review (2)

Comment thread eth/tracers/api_test.go Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Core state-transition routing and fork-sensitive replay paths require final human validation.

Review effort: Balanced
Findings: None

Resolved since last review (1)

@gzliudan gzliudan changed the title fix(core,eth): fix nonce too low and wrong intermediate roots in debug tracing fix(core,eth,miner): fix nonce too low, wrong intermediate roots and the traces of the TIPSigning activation block Sep 19, 2026
…ing activation block

Block processing removes the legacy block signers account before the first
transaction of the block that activates TIPSigning, but none of the replays did
it: stateAtTransaction, debug_intermediateRoots and the blocks debug_traceChain
walks rebuild the pre-state of that block from the parent state, so the
transaction traces and every intermediate root of the activation block describe
a state the chain never had, one that still holds an account canonical execution
had already removed.

Share the removal as core.ApplyTIPSigningHardFork, call it from the three replay
paths, and replace the three copies in Process, ProcessBlockNoValidator and the
miner with it. The condition fires on the activation block only and all three
copies already agreed on it, so block processing and the miner build the same
block as before.

The chain makers had the same gap: they mirror the block level mutations of the
DAO fork and of Prague, so a generated chain that activates TIPSigning inside it
produces blocks whose root does not match what block processing computes, and
such a block cannot be imported at all. They now run the shared helper too.

Add TestIntermediateRootsMatchesBlockProcessingAtTIPSigningActivation, which
traces a block that activates TIPSigning over a parent state holding the legacy
account, pins that premise from both sides (the account is in the parent state
and gone from the state the imported block carries) and compares the
intermediate roots against a block processing replay pinned to block.Root();
without the call the very first root already differs.

This is a pre-existing gap: the base commit replays those blocks through
ApplyMessage and without the removal as well. geth has no TIPSigning fork, so
there is no upstream fix to port.

TestIntermediateRootsMatchesBlockProcessingAtTIPSigningActivation carries its own
mirror of the block processing replay instead of building on the helper of the
intermediate-roots fix, so this commit can be reviewed and imported on its own.
@gzliudan gzliudan changed the title fix(core,eth,miner): fix nonce too low, wrong intermediate roots and the traces of the TIPSigning activation block fix(core,eth,miner): fix traces and intermediate roots of the TIPSigning activation block Sep 19, 2026
@gzliudan gzliudan closed this Sep 19, 2026
@gzliudan
gzliudan deleted the fix-trace-replay branch September 19, 2026 10:12
@gzliudan

Copy link
Copy Markdown
Collaborator Author

Branch renamed to fix-trace-replay-tipsigning, which closed this pull request: the head branch lives in the gzliudan fork, and GitHub does not carry a head rename made in a fork over to the pull request. The change is rebuilt as #2591 with the identical head commit facfa9374, so please continue the review there. The review threads above stay here for reference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants