Skip to content

perf: deserialize DKG messages once (framing-only intake) - #7557

Merged
PastaPastaPasta merged 3 commits into
dashpay:developfrom
PastaPastaPasta:perf/dkg-intake-single-deserialize
Aug 15, 2026
Merged

perf: deserialize DKG messages once (framing-only intake)#7557
PastaPastaPasta merged 3 commits into
dashpay:developfrom
PastaPastaPasta:perf/dkg-intake-single-deserialize

Conversation

@PastaPastaPasta

@PastaPastaPasta PastaPastaPasta commented Aug 7, 2026

Copy link
Copy Markdown
Member

Issue being fixed or feature implemented

DKG network message intake deserializes each accepted payload twice: once on a copied payload in the p2p message handler for structural validation, and again from the pending queue on the DKG worker thread. These messages carry BLS objects, so the redundant intake pass repeats elliptic-curve point decompression on the shared network thread during every DKG round.

Additionally, the per-peer pending-message quota in the same path is keyed by NodeId, so a single misbehaving masternode can reset its retention budget just by reconnecting, defeating the bound that already exists.

This is a from-scratch redesign of the approach in #7401, sharing its goals and test strategy but with a substantially smaller intake parser and a simpler retention model.

What was done?

  • Incoming contributions, complaints, justifications, and premature commitments are retained as exact raw wire bytes in the existing per-message-type pending queues.
  • The typed intake deserialize is replaced by CheckDKGMessageWireStructure(), a framing-only walk that validates CompactSize counts, dynamic bitsets, quorum-parameter bounds, truncation, and trailing bytes without decoding any BLS object. The walk is deliberately thin: fixed-size BLS encodings are skipped with a single bounds-checked ignore(), and dynamic bitsets are validated by calling the same ReadFixedBitSet() the typed deserializer uses, so truncation and padding-bit semantics cannot diverge.
  • The common llmqType/quorumHash prefix is peeked via SpanReader instead of read-then-Rewind, and short payloads are rejected up front instead of throwing out of ProcessMessage.
  • The DKG worker is the sole typed deserialization point (BLS decompression, canonical checks, active-scheme handling happen exactly once), immediately followed by the same parameter-derived structural checks and normal preverification. Failures are scored 100 as before.
  • The pre-existing per-peer retention quota (twice the quorum size, per message type) is now keyed by proTxHash (maxMessagesPerProTx) instead of NodeId, so reconnecting with a fresh NodeId no longer resets the budget, and the quota is cumulative for the round (draining the queue does not refund it). Own messages are enqueued under this node's own proTxHash and go through the same quota path -- no special case.
  • Threat model note: sender identities are pinned to the deterministic masternode list by MNAuth, so each additional quota is gated by masternode collateral. Under the assumption that only a small number of masternodes are malicious, the per-proTx quota alone bounds worst-case retention to (hostile MN count) x (2 x quorum size) messages per type; no queue-wide cap is introduced.
  • Duplicate wire hashes are rejected before charging the quota; messages dropped for quota are not marked seen, so they can be announced and delivered again by a peer with remaining budget.
  • Inventory hashes are computed over the exact original wire bytes; own-message queueing and malformed-message scoring are preserved.
  • At round start, leftover raw queues are discarded without any typed or BLS deserialization, so stale messages cannot delay next-round initialization.
  • Deliberate strengthening vs. develop worth calling out: trailing bytes after a structurally complete message are now rejected (at intake by the framing walk, and defensively on the worker). Previously neither pass checked for them, so payload || garbage was accepted and hashed as a distinct inventory item.
  • Added a fuzz target (dkg_message_framing) that continuously checks the safety-critical equivalence direction across every configured LLMQ and both BLS schemes: the framing walk must never reject a payload that typed worker deserialization would accept. The converse is intentionally not asserted -- framing accepts undecodable BLS encodings so the worker can score the sender.
  • Unit tests pin the quota semantics (reconnect persistence, no refund on drain, dedup-before-quota, per-proTx independence, own messages sharing the quota path). Functional tests cover trailing-byte rejection at intake, deferred BLS scoring on the worker, proTxHash-keyed quotas across reconnects, and late-message retention cleared at round start without BLS decoding.

How Has This Been Tested?

Validated on macOS arm64 with:

make -C src -j15 dashd test/test_dash
./src/test/test_dash --run_test=llmq_dkg_tests --catch_system_errors=no
test/functional/test_runner.py feature_llmq_dkg_intake.py
FUZZ=dkg_message_framing ./src/test/fuzz/fuzz <seeds>  # 402-seed replay
python3 test/lint/lint-includes.py
python3 test/lint/lint-circular-dependencies.py
python3 test/lint/lint-python.py
test/lint/lint-whitespace.py

Breaking Changes

None.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation
  • I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b191232c-87b8-4a38-8d3a-dd94e8e9752e

📥 Commits

Reviewing files that changed from the base of the PR and between 8aa661a and 5d46561.

📒 Files selected for processing (4)
  • src/Makefile.test.include
  • src/llmq/net_dkg.cpp
  • src/llmq/net_dkg.h
  • test/util/data/non-backported.txt
🚧 Files skipped from review as they are similar to previous changes (4)
  • test/util/data/non-backported.txt
  • src/Makefile.test.include
  • src/llmq/net_dkg.h
  • src/llmq/net_dkg.cpp

Walkthrough

DKG intake now validates bounded wire framing without consuming the receive stream or decoding BLS objects. Typed deserialization and structural validation occur during worker processing. Observer nodes ignore and do not request DKG round payloads. Pending-message quotas use cumulative sender proTxHash accounting across reconnects. New fuzz and functional tests cover framing, deferred failures, observer handling, quotas, and round cleanup.

Estimated code review effort: 4 (Complex) | ~60 minutes

Merge Risk: ⚪ Minimal · up to 5d465

The PR changes DKG message intake and retention behavior, but no actionable merge-blocking risk remains in the supplied evidence; it is merge-ready after normal checks and review.

Sequence Diagram(s)

sequenceDiagram
  participant Peer
  participant NetDKG
  participant CDKGPendingMessages
  participant DKGWorker
  Peer->>NetDKG: send DKG payload
  NetDKG->>NetDKG: validate wire framing
  NetDKG->>CDKGPendingMessages: retain raw payload with sender proTxHash
  DKGWorker->>CDKGPendingMessages: retrieve pending payload
  DKGWorker->>DKGWorker: deserialize and structurally validate
  DKGWorker->>DKGWorker: preverify and process message
Loading

Possibly related PRs

  • dashpay/dash#7401: Implements the same DKG intake hardening, quotas, framing validation, deferred deserialization, fuzz target, and functional tests.
  • dashpay/dash#7524: Also modifies DKG pending-message retention across reconnecting peer IDs.
  • dashpay/dash#7583: Extends the per-proTxHash DKG pending-message quota changes in the same classes and tests.

Suggested reviewers: udjinm6

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the primary change: deferring DKG deserialization through framing-only intake.
Description check ✅ Passed The description accurately explains the framing redesign, deferred deserialization, quota changes, tests, and implementation objectives.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@thepastaclaw

thepastaclaw commented Aug 7, 2026

Copy link
Copy Markdown

🕓 Ready for review — next in queue (commit 5d46561)
Queue position: 1/1
ETA: start ~01:10 UTC · complete ~01:21 UTC (median 11m across 30 recent reviews; 2 slots)
Queued 18m ago · Last checked: 2026-08-15 01:10 UTC

@PastaPastaPasta
PastaPastaPasta force-pushed the perf/dkg-intake-single-deserialize branch 2 times, most recently from 8389144 to 11cd6c4 Compare August 7, 2026 20:43

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preliminary review — Codex only

The framing parser and proTxHash-keyed quota redesign are well covered, but moving malformed BLS detection to the DKG worker creates a deterministic penalty bypass when the originating peer disconnects before its queued message is processed, so changes are required. The new Dash-specific fuzz target must also be registered in the non-backported manifest to receive the intended lint coverage.
Source: Reviewer backend models: codex general gpt-5.6-sol and codex dash-core-commit-history gpt-5.6-sol; final verifier backend model: gpt-5.6-sol. openclaw-agent/cliproxy/gpt-5.6-sol is orchestration-only and not reviewer evidence.

Validated blockers were found in the Codex precheck. Opus is deferred until a fresh Codex revalidation clears the blocker gate.

Review provenance

  • Codex reviewers: gpt-5.6-sol — general (completed), gpt-5.6-sol — dash-core-commit-history (completed)
  • Verifier: gpt-5.6-sol — verifier
  • Sonnet: not run (deferred by blocker gate)

🔴 1 blocking | 🟡 1 suggestion(s)

🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `src/llmq/net_dkg.cpp`:
- [BLOCKING] src/llmq/net_dkg.cpp:403-407: Deferred BLS failures lose their penalty after reconnect
  The queued entry retains only the originating `NodeId`, while this PR moves malformed BLS detection from synchronous intake validation to the later DKG worker pass. A peer can send a requested, framing-valid payload with an invalid BLS encoding during `Initialized`, disconnect before the matching phase drains the queue, and reconnect under a new ID. `FinalizeNode()` removes the old ID from `PeerManagerImpl::m_peer_map`, so the later `PeerMisbehaving(nodeId, 100)` call finds no `PeerRef` and silently applies no score. This is a regression from the previous typed intake check, which detected this malformed encoding while the sender was still being processed. The proTxHash-keyed quota limits retained work but does not preserve punishment; retain enough authenticated sender metadata to apply the offense after disconnect, or otherwise ensure deferred validation keeps the originating peer punishable.

In `src/test/fuzz/dkg_message_framing.cpp`:
- [SUGGESTION] src/test/fuzz/dkg_message_framing.cpp:1: Register the new Dash-specific fuzz file as non-backported
  `src/test/fuzz/dkg_message_framing.cpp` is a newly added Dash-specific source, but no pattern in `test/util/data/non-backported.txt` matches it; `src/test/llmq*.cpp` only covers files directly under `src/test`. The manifest feeds Dash-specific cppcheck and clang-diff-format coverage, so add the fuzz target's exact path to it.

Comment thread src/llmq/net_dkg.cpp
Comment thread src/test/fuzz/dkg_message_framing.cpp
@PastaPastaPasta

Copy link
Copy Markdown
Member Author

Thanks for the review. Taking the two findings in turn:

Suggestion (non-backported.txt): accepted. src/test/fuzz/dkg_message_framing.cpp is now registered in test/util/data/non-backported.txt (verified git ls-files resolves it), so it gets Dash-specific cppcheck and clang-format-diff coverage.

Blocker (deferred BLS penalty lost after reconnect): declining. The mechanics of the finding are accurate — PeerMisbehaving no-ops once FinalizeNode() has run, so a peer that disconnects before the phase drain escapes the score for a malformed BLS encoding. But this does not rise to a blocker:

  1. The baseline being "regressed" is six weeks old, and the historical behavior is what this PR restores. Synchronous typed intake validation was introduced in 31142da (merged 2026-06-29). From the DKG's introduction until then, malformed BLS encodings were detected exactly where this PR detects them — on the DKG worker via PopAndDeserializeMessages returning nullptr — with the identical disconnect-evasion window. That commit's purpose was preventing unauthenticated retention and crashes; those protections are kept and strengthened here (MNAuth gate, framing walk, size caps, per-proTx quota).

  2. The same window already exists — before and after this PR — for the sibling failure classes on this exact code path. A well-encoded message with a bad signature, or one failing PreVerifyMessage, is only punished at drain time (ProcessPendingMessageBatch / BatchVerifyMessageSigs), and always has been. The invariant "the sender is still connected when punished" was never held by deferred DKG validation in general; singling out the encoding-failure class doesn't change the security posture, because an attacker who wants to dodge the score can already do so today by sending a framing-valid, signature-invalid payload and disconnecting.

  3. What the score protects is bounded by a mechanism that does survive reconnect. The DoS-relevant resources — queue retention and worker-side BLS decompression — are capped by the proTxHash-keyed quota this PR introduces, which is deliberately cumulative for the round, not refunded on pop, and unaffected by disconnect/reconnect. Worst case per masternode identity per phase is 2 × quorumSize framing-valid payloads decoded once each. Reaching the queue at all requires an MNAuth-verified, on-chain-registered masternode identity, which is Sybil-expensive; the peer score is a secondary, best-effort deterrent on top of that, as it is everywhere else in the codebase.

  4. The proposed fix has worse semantics than the gap. Retaining sender identity and punishing "whoever currently holds that proTxHash" would discourage a fresh, MNAuth-verified connection for bytes attributed to a previous session — effectively reimplementing PoSe at the peer layer with confusing address-discouragement side effects. Identity-level accountability for DKG misconduct already lives at the session/PoSe layer, which is keyed by the durable identity and is the right place for it.

If reviewers feel the residual gap is worth tracking, a follow-up issue for protx-keyed misbehavior accounting across the deferred DKG validation paths (all of them, not just decode failures) would be the coherent shape — but it is orthogonal to this PR's perf change and predates it.


🤖 Posted autonomously by Claude on behalf of pasta.

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Codex + Sonnet

The framing-only intake parser, worker-side typed validation, proTxHash-keyed cumulative quotas, and associated fuzz/unit/functional coverage are consistent at the exact head. The prior non-backported manifest omission is fixed, and the deferred peer-scoring disconnect window is bounded by this PR's durable quota and remains a broader pre-existing behavior rather than an in-scope blocker; no actionable findings remain.
Source: Reviewer backend models gpt-5.6-sol (Codex general and dash-core-commit-history) and claude-sonnet-5 (Claude general and dash-core-commit-history); final verifier backend model gpt-5.6-sol as the permitted fallback after both Claude Sonnet verifier attempts failed. openclaw-agent/cliproxy/gpt-5.6-sol is orchestration-only and not reviewer evidence.

Review provenance

  • Codex reviewers: gpt-5.6-sol — dash-core-commit-history (completed), gpt-5.6-sol — general (completed)
  • Verifier: gpt-5.6-sol — final-verifier (fallback for Sonnet verifier)
  • Sonnet reviewers: claude-sonnet-5 — general (completed), claude-sonnet-5 — dash-core-commit-history (completed)

@thepastaclaw

Copy link
Copy Markdown

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 9, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
test/functional/feature_llmq_dkg_intake.py (1)

260-268: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Remove the unused node parameter.

_send_late_qcontrib does not use node.

♻️ Proposed change
-    def _send_late_qcontrib(self, node, peer, nonce):
+    def _send_late_qcontrib(self, peer, nonce):

Update the three call sites in test_late_messages_bounded accordingly.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/functional/feature_llmq_dkg_intake.py` around lines 260 - 268, Remove
the unused node parameter from _send_late_qcontrib and update all three calls in
test_late_messages_bounded to pass only peer and nonce.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/test/llmq_dkg_tests.cpp`:
- Line 51: Update all three CDKGPendingMessages constructor calls in the test to
use the declared parameter name _maxMessagesPerProTx in their argument comments,
or remove the comments entirely.

---

Nitpick comments:
In `@test/functional/feature_llmq_dkg_intake.py`:
- Around line 260-268: Remove the unused node parameter from _send_late_qcontrib
and update all three calls in test_late_messages_bounded to pass only peer and
nonce.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 835a2cc1-8eb6-4bba-87a0-67c4162b4173

📥 Commits

Reviewing files that changed from the base of the PR and between 4dfe036 and 5d06f2e.

📒 Files selected for processing (9)
  • src/Makefile.test.include
  • src/llmq/dkgsessionhandler.cpp
  • src/llmq/dkgsessionhandler.h
  • src/llmq/net_dkg.cpp
  • src/llmq/net_dkg.h
  • src/test/fuzz/dkg_message_framing.cpp
  • src/test/llmq_dkg_tests.cpp
  • test/functional/feature_llmq_dkg_intake.py
  • test/util/data/non-backported.txt

Comment thread src/test/llmq_dkg_tests.cpp Outdated

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preliminary review — Codex only

The active-masternode path has focused framing validation, worker-side typed validation, and strong quota and fuzz coverage, but observer mode retains accepted payloads without ever running the deferred validation or round cleanup. Two commit-history cleanup suggestions also remain.
Source: Reviewer backend models: gpt-5.6-sol (Codex general and dash-core-commit-history); final verifier backend model: gpt-5.6-sol. openclaw-agent/cliproxy/gpt-5.6-sol is orchestration-only and not reviewer evidence.

Validated blockers were found in the Codex precheck. Opus is deferred until a fresh Codex revalidation clears the blocker gate.

Review provenance

  • Codex reviewers: gpt-5.6-sol — general (completed), gpt-5.6-sol — dash-core-commit-history (completed)
  • Verifier: gpt-5.6-sol — verifier
  • Sonnet: not run (deferred by blocker gate)

🔴 1 blocking | 🟡 2 suggestion(s)

2 additional finding(s) omitted (not in diff).

🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `src/llmq/net_dkg.cpp`:
- [BLOCKING] src/llmq/net_dkg.cpp:591-595: Observer mode has no worker to perform the deferred BLS validation
  This framing-only check is also used by watch-only observer instances, but those instances have no path that performs the promised deferred typed validation. The observer constructor installs plain `CDKGSessionHandler` objects, `NetDKG::Start()` returns immediately when `m_active` is null, and `ClearPendingMessages()` is only called by the active-only `HandleDKGRound()`. A requested payload with framing-valid but invalid BLS encodings is therefore retained indefinitely without calling `DeserializeCheckedDKGMessage()`, scoring its sender, or clearing its bytes, seen hash, and proTxHash quota. Because membership validation is likewise worker-side, any MNAuth-verified masternode can fill these per-handler queues even when it is not a member of the targeted quorum. This regresses the previous synchronous typed intake rejection for malformed BLS encodings and makes the new nominally per-round proTxHash quota process-lifetime state for observers. Keep typed validation in observer mode, avoid retaining DKG payloads there, or add an observer lifecycle that validates or clears every queue at round boundaries.

In `test/util/data/non-backported.txt`:
- [SUGGESTION] test/util/data/non-backported.txt:1: Squash the manifest correction into the fuzz-target commit
  Commit `5d06f2e30f44` only adds the required non-backported manifest entry for `src/test/fuzz/dkg_message_framing.cpp`, which commit `863d53f08696` introduced. This is part of making that new Dash-specific target complete rather than a separate logical change. Squash `5d06f2e30f44` into `863d53f08696` so the fuzz target enters history with its required lint registration.

In `<commit:5f3ec10b2df>`:
- [SUGGESTION] <commit:5f3ec10b2df>:1: Fold the review-comment cleanup into the test commit
  Commit `5f3ec10b2dff` consists solely of corrections to tests introduced by `11cd6c4cb1a4`: it aligns three argument comments with the constructor parameter and removes an unused Python helper parameter. Its subject depends on temporary review context and does not represent an independent logical change. Fold it into `11cd6c4cb1a4` so the tests enter permanent history in their final form.

Comment thread src/llmq/net_dkg.cpp
@PastaPastaPasta
PastaPastaPasta force-pushed the perf/dkg-intake-single-deserialize branch from 5f3ec10 to 8aa661a Compare August 10, 2026 14:05
@PastaPastaPasta

Copy link
Copy Markdown
Member Author

Applied the two commit-history suggestions as part of the force-with-lease update:

  • test/util/data/non-backported.txt is now part of the fuzz-target commit 446c7c212084.
  • The argument-comment and unused-helper-parameter cleanup is now part of the test commit 8aa661a14f12.

The PR now has three self-contained commits. The final tree is unchanged apart from the observer-mode fix and regression coverage, and passed the DKG unit suite plus feature_llmq_dkg_intake.py locally.


🤖 Posted autonomously by Codex on behalf of pasta.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/functional/feature_llmq_dkg_intake.py`:
- Line 130: Update the extra_args construction to use iterable unpacking instead
of list concatenation, preserving one entry with “-watchquorums=1” and three
entries containing common_args while resolving Ruff RUF005.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 192e95e0-779e-4190-8ee4-7eecc6261f21

📥 Commits

Reviewing files that changed from the base of the PR and between 5d06f2e and 8aa661a.

📒 Files selected for processing (4)
  • src/llmq/net_dkg.cpp
  • src/llmq/net_dkg.h
  • src/test/llmq_dkg_tests.cpp
  • test/functional/feature_llmq_dkg_intake.py
🚧 Files skipped from review as they are similar to previous changes (3)
  • src/test/llmq_dkg_tests.cpp
  • src/llmq/net_dkg.h
  • src/llmq/net_dkg.cpp

Comment thread test/functional/feature_llmq_dkg_intake.py

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Codex/Sol only (Phase 2 disabled)

The observer-mode retention defect and both prior commit-history issues are fixed at the exact head. One non-blocking commit-message issue remains: the production and test commits omit the newly added observer-mode behavior and regression coverage. The CodeRabbit RUF005 comment is not applicable to this repository's flake8/mypy lint configuration.
Source: Reviewer backend models: gpt-5.6-sol (Codex general and dash-core-commit-history); final verifier backend model: gpt-5.6-sol. openclaw-agent/cliproxy/gpt-5.6-sol is orchestration-only and not reviewer evidence.

Validated zero-blocker Codex/Sol precheck evidence was promoted to final because Phase 2 (Sonnet/Opus) is temporarily disabled. This is Codex/Sol-only final validation, not Codex + Sonnet/Opus coverage.

Review provenance

  • Codex reviewers: gpt-5.6-sol — general (completed), gpt-5.6-sol — dash-core-commit-history (completed)
  • Verifier: gpt-5.6-sol — verifier
  • Sonnet/Opus: not run (Phase 2 disabled — temporary Codex/Sol-only final)
  • Secondary pass: disabled (temporary_phase2_sonnet_disable)

🟡 1 suggestion(s)

1 additional finding(s) omitted (not in diff).

🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `<commit:6aa2f7833b6>`:
- [SUGGESTION] <commit:6aa2f7833b6>:1: Document the observer-mode behavior in the commit messages
  Commit `6aa2f7833b6` now makes observer nodes report DKG round inventory as already known and ignore directly delivered DKG payloads because they have no worker to validate or drain them. This is a substantive part of making deferred deserialization safe, but the production commit message only says that the worker becomes the sole typed-deserialization point. Commit `8aa661a14f1` also adds the corresponding observer regression test without mentioning it in its detailed test summary. Amend the production commit message to explain why observers no longer request or retain round payloads, and mention the observer coverage in the test commit message.

@PastaPastaPasta

Copy link
Copy Markdown
Member Author

Per the release-planning discussion, the DoS-relevant part of this PR (the per-proTxHash pending-message bound) has been extracted into #7583 to ship in the release. The framing-only intake / deserialize-once redesign here is deferred to post-RC; converting to draft until then. This branch will be rebased on top of #7583 once that merges.


🤖 Posted autonomously by Claude on behalf of pasta.

PastaPastaPasta added a commit to PastaPastaPasta/dash that referenced this pull request Aug 12, 2026
Unit tests: the quota is charged per proTxHash (surviving NodeId changes and not refunded on pop), duplicates are rejected before charging the quota, distinct proTxHashes have independent budgets, and locally produced messages share the quota path.

Functional test: extend feature_llmq_dkg_intake.py with a late-message scenario proving that a masternode identity reconnecting under fresh NodeIds cannot retain more than maxMessagesPerProTx contributions, that quota drops are silent (banscore stays 0), that a distinct proTx keeps its own budget, and that round-start clearing discards retained messages without them ever reaching a worker.

Ported from dashpay#7557, adapted to the pre-framing intake (well-formed zero-BLS payloads instead of BLS-invalid ones, since develop still deserializes a copy at intake).
PastaPastaPasta added a commit that referenced this pull request Aug 12, 2026
The DKG pending-message retention quota was keyed by NodeId, which resets on reconnect: a single masternode identity could retain an unbounded number of pending messages across reconnects. Key the quota by the sender's proTxHash instead, which survives reconnects and is pinned to registered masternode identities by the MNAuth gate (develop already rejects pushed DKG messages from peers without a verified proRegTxHash). The quota is cumulative for the round and not refunded on pop, so draining the queue does not regain retention slots.

Also check the duplicate-hash set before charging the quota so resent hashes don't burn budget, and do not mark a quota-dropped hash as seen so another peer with remaining budget can re-deliver it. Locally produced messages (from=-1) are enqueued under this node's own proTxHash and charged like any other sender's.

Extracted from the DKG intake redesign in #7557; the deserialize-once/framing changes there are deliberately not included.
PastaPastaPasta added a commit that referenced this pull request Aug 12, 2026
cf85283 test: cover per-proTx DKG pending-message quotas (pasta)
4b7e21f fix: bound pending DKG message retention per proTxHash (pasta)

Pull request description:

  ## Issue being fixed or feature implemented

  The DKG pending-message retention quota in `CDKGPendingMessages` is keyed by NodeId, which resets on every reconnect. A single masternode identity can therefore retain an effectively unbounded number of pending DKG messages on a victim masternode by reconnecting between sends.

  This extracts the DoS-relevant part of #7557 so it can ship in the release; the deserialize-once/framing intake redesign remains in #7557 for post-RC.

  ## What was done?

  - Rekeyed the retention quota to the sender's MNAuth-verified proTxHash (locally produced messages charge this node's own proTxHash). The quota is cumulative for the round and not refunded on pop, so neither reconnects nor queue drains regain retention slots. The MNAuth gate pins quota keys to registered masternode identities.
  - The duplicate-hash check now runs before the quota charge so resent hashes don't burn budget, and quota-dropped hashes are not marked seen, so another peer with remaining budget can re-deliver the message.
  - `PopAndDeserializeMessages` and the framing/deserialize-once changes from #7557 are intentionally not included.

  ## How Has This Been Tested?

  - New unit tests in `llmq_dkg_tests`: quota survives NodeId changes, quota is per-proTx, duplicates are rejected before charging the quota, own-message enqueue path shares the quota.
  - Extended `feature_llmq_dkg_intake.py` with a late-message scenario: a reconnecting identity cannot exceed its budget across `2*llmq_size` fresh NodeIds, over-quota drops are silent (banscore 0), a distinct proTx has its own budget, and round-start clearing discards retained messages without scoring.
  - `make -j5`, `test_dash --run_test=llmq_dkg_tests`, `test_runner.py feature_llmq_dkg_intake.py`, `lint-python`, `lint-whitespace`.

  ## Breaking Changes

  None. Honest masternodes send one message per type per round, far under the unchanged `size * 2` budget; the only behavior change is that a reconnecting identity can no longer refill its retention quota.

  ## Checklist:

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

Top commit has no ACKs.

Tree-SHA512: cf881cef9465b8b37f31b9630c937d35c40024ed86771d20d717b4a09a06b9c2467662d05c50a8b4a1f92b96d62dceafe565478eafb6f35610f2f994ef6e2ce0
@github-actions

Copy link
Copy Markdown

This pull request has conflicts, please rebase.

PastaPastaPasta and others added 3 commits August 12, 2026 12:30
DKG intake previously deserialized a copy of each accepted payload (repeating BLS point decompression on the shared network thread) for structural validation, and the DKG worker then deserialized the retained bytes again. Replace the typed intake pass with a framing-only wire walk that validates CompactSize counts, dynamic bitsets (via the same ReadFixedBitSet the typed path uses), quorum-parameter bounds, truncation, and trailing bytes without decoding any BLS object. The worker is now the sole typed deserialization point, immediately followed by the same parameter-derived structural checks.

The pre-existing per-peer pending-message quota is rekeyed from NodeId to the MNAuth-verified proTxHash and made cumulative for the round, so a sender can no longer reset its retention budget by reconnecting or by waiting for the worker to drain the queue. Own messages are enqueued under this node's own proTxHash and share the same quota path. Sender identities are pinned to the deterministic masternode list by MNAuth, so worst-case retention is bounded by (hostile MN count) x quota. Duplicate hashes are rejected before charging the quota, and quota-dropped messages are not marked seen so another peer with budget can re-deliver them. The llmqType/quorumHash prefix is peeked via SpanReader instead of read+Rewind, and short payloads are scored instead of throwing out of ProcessMessage. Leftover raw queues are discarded at round start without BLS work.

Co-Authored-By: Claude Fable 5 <[email protected]>
Intake framing validation and worker typed deserialization are two hand-maintained parsers over one wire format. The safety-critical direction is that framing must never reject a payload the worker would accept, otherwise honest DKG messages are silently dropped before retention and quorum formation degrades. Assert that direction over fuzzer-provided payloads for every configured LLMQ and both BLS schemes, plus a constructed well-formed message per input so serializer/framing drift is caught even from an empty corpus. The converse is intentionally not asserted: framing accepts undecodable BLS encodings so the worker can score the sender.

Co-Authored-By: Claude Fable 5 <[email protected]>
Unit tests pin the CDKGPendingMessages semantics: the per-proTx quota survives reconnects and is not refunded by drains, duplicates are rejected before charging, quotas are independent across proTxes, and own messages are charged under this node's own proTxHash. Functional tests cover trailing-byte rejection at intake, deferral of BLS decoding to the DKG worker (scored there, not at intake), quota persistence across reconnects under fresh NodeIds, and late-message retention cleared at round start without BLS work.

Co-Authored-By: Claude Fable 5 <[email protected]>
@PastaPastaPasta
PastaPastaPasta force-pushed the perf/dkg-intake-single-deserialize branch from 8aa661a to 5d46561 Compare August 12, 2026 17:39
@UdjinM6

UdjinM6 commented Aug 13, 2026

Copy link
Copy Markdown

From Claude:

1. net_dkg.cpp:511 — observer mode no longer scores pushed DKG messages at all

The m_active == nullptr early return sits ahead of the MNAuth gate and every
later check, so a -watchquorums node loses the whole DKG-specific banscore
path: +10 for a non-MNAuth sender, +100 for oversized/malformed, +10 for
unrequested. Generic transport limits still apply, but nothing DKG-aware
discourages a peer that pushes these continuously. It also makes observer mode
more permissive than NetDKGStub::ProcessMessage (line 936), which still
misbehaves 10 for exactly these types.

Since AlreadyHave now returns true unconditionally in observer mode, an
observer never sends getdata for these, and the only production sends of
qcontrib/qcomplaint/qjustify/qpcommitment are ProcessGetData replies —
so everything arriving here is unsolicited by construction and can be scored
without risk to honest peers:

if (m_active == nullptr) {
    LogPrint(BCLog::LLMQ_DKG, "NetDKG -- ignoring %s in observer mode\n", msg_type);
    m_peer_manager->PeerMisbehaving(pfrom.GetId(), UNREQUESTED_OBJECT_MISBEHAVIOR_SCORE,
                                    "unrequested DKG message");
    return;
}

Same constant the unrequested path already uses at line 636.
feature_llmq_dkg_intake.py:199 would need its expected banscore bumped from 0
to 10.

2. net_dkg.cpp:675 — observers still attract DKG invs they can no longer use

ObserverContext::IsWatching() is hardcoded true, so the observer keeps
sending QWATCH, and masternodes keep PeerPushInventory-ing every round inv to
it via RelayInvToParticipants (line 341). AlreadyHave silently suppresses the
getdata, so these are one-way invs dropped without a request — unchanged traffic
from develop for payloads that are now never used.

Dropping the advertisement isn't an option: qwatch also gates QGETDATA on the
serving masternode (net_quorum.cpp:74 scores +10 without it, since an observer
has no verified proTxHash) and the connection-retention exception
(masternode/utils.cpp:65). So either add a comment recording that the invs are
the price of the quorum-data channel, or filter sender-side:

pnode->qwatch && !pnode->GetVerifiedProRegTxHash().IsNull()

A pure observer is never MNAuth'd, while a masternode running -watchquorums
always is — PushMNAUTH isn't gated on quorum membership and is queued ahead of
QWATCH in the same VERACK handler (net_processing.cpp:4082 vs :4105), so
the verified hash is set before qwatch is.

Minor

DKG_MSG_PREFIX_SIZE (line 77), the local PREFIX (line 46), and the
sizeof(uint8_t) + sizeof(uint256) guard (line 531) are three spellings of the
same wire constant, in a file whose stated risk model is parser drift.

@PastaPastaPasta

Copy link
Copy Markdown
Member Author

observer mode no longer scores pushed DKG messages at all

Do we care? this is a dev mode really that no-one likely runs. Do we care if in dev mode, we don't ban people?

@UdjinM6

UdjinM6 commented Aug 14, 2026

Copy link
Copy Markdown

observer mode no longer scores pushed DKG messages at all

Do we care? this is a dev mode really that no-one likely runs. Do we care if in dev mode, we don't ban people?

Fair. More like a nit then.

@UdjinM6 UdjinM6 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK 5d46561

@PastaPastaPasta
PastaPastaPasta marked this pull request as ready for review August 15, 2026 00:48
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@PastaPastaPasta
PastaPastaPasta merged commit 3b865cf into dashpay:develop Aug 15, 2026
73 of 76 checks passed
@PastaPastaPasta
PastaPastaPasta deleted the perf/dkg-intake-single-deserialize branch August 15, 2026 04:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants