Skip to content

feat(activitypub): double-knock delivery signatures and per-host circuit breaker - #574

Merged
davidwkeith merged 3 commits into
mainfrom
claude/trusting-hawking-3eawag
Oct 28, 2026
Merged

davidwkeith merged 3 commits into
mainfrom
claude/trusting-hawking-3eawag

Conversation

@davidwkeith

@davidwkeith davidwkeith commented Oct 27, 2026 •

Copy link
Copy Markdown
Owner

Summary

Two delivery-hardening patterns from SiliconBeest and Fedify. State for both lives in the actor Durable Object's SQLite, not KV, per the consistency rules.

Double-knock signatures

  1. Each delivery is signed with RFC 9421 first (@method @target-uri content-digest content-type, created, rsa-v1_5-sha256), using the new signRequestRfc9421, which sits on @dwk/http-signatures.
  2. If the peer answers 400/401/403, the same body is re-signed with draft-cavage and sent once more. Other statuses (404/410/422/5xx) never trigger a second knock. A peer that blocks us with 403 gets two requests per delivery. This is accepted and documented next to KNOCK_AGAIN_STATUSES.
  3. When the second knock succeeds, the host is stored in a new sig_pref table. For 7 days after that, deliveries to the host skip the RFC 9421 knock. The row then expires and is pruned, so a peer that adds RFC 9421 support later gets found again.

Outbound GETs (authorized-fetch key resolution) stay on draft-cavage. API changes: deliverActivity's optional trailing now/timeoutMs arguments move into a DeliveryOptions object, which also carries preference. DeliveryResult now includes the accepted profile on success.

Per-host circuit breaker (peer_circuit)

  • Five consecutive retryable failures to a host (network error, 5xx, 408, 429), counted across all of its delivery rows, open the circuit for 10 minutes. While it is open, that host's due rows are deferred to the end of the cooldown and are not charged a retry attempt, so one dead peer no longer spends each alarm wake on 10 s timeouts.
  • When the cooldown elapses, the next row is a half-open probe. A failure re-opens the circuit with double the previous cooldown, capped at 24 h. A success closes it.
  • Any non-retryable answer also closes the circuit, as does a validly-signed inbound activity from the host (Mastodon handles unavailable domains the same way).
  • Once a host has been failing for 7 days (Mastodon's unavailable-domain window), rows that reach it while its circuit is open are dropped. One probe per cooldown still goes out.
  • New events: activitypub.delivery.deferred, activitypub.delivery.circuit_opened, activitypub.delivery.circuit_closed. A drop is logged as delivery.failed with reason: "circuit_open".
  • The thresholds are constants for now, not config.

Packages affected

@dwk/activitypub (plus updated assertions in @dwk/server's activitypub integration tests)

Checklist

  • Read the relevant spec(s) under spec/packages/ and updated them if
    behaviour changed: spec/packages/activitypub.md S2S section (both features)
  • Added/updated colocated tests (src/*.test.ts). Double-knock: fallback on 401, skipping the first knock for a known draft-cavage peer, no second knock on 404/410/422/500, a round trip through our own RFC 9421 verifier, and the preference stored, used, then expired. Circuit breaker: opens after 5 failures with the remaining rows deferred and not charged attempts, a failed probe doubles the cooldown, a successful probe delivers the backlog, a 7-day-dead host's rows are dropped, and an inbound signed activity closes the circuit.
  • Ran the local CI gate: pnpm lint && pnpm format:check && pnpm typecheck && pnpm build && pnpm test (3284 passed)
  • Added a changeset (pnpm changeset) if this touches a publishable
    package (one per feature)
  • Updated catalog.json / conformance/status.json if this adds a new
    mountable worker or changes conformance status: not applicable, no new worker and no change to conformance status

🤖 Generated with Claude Code

https://claude.ai/code/session_015LRGjmHE99WESN5HBERxSj

Sign each outbound inbox POST with RFC 9421 first and, when the peer
answers 400/401/403, re-sign the same body with draft-cavage and send
once more. Hosts that only accept draft-cavage are remembered in the
actor DO's SQLite (`sig_pref`) for 7 days so later deliveries skip the
first knock. Outbound GETs stay draft-cavage.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_015LRGjmHE99WESN5HBERxSj

@davidwkeith davidwkeith left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

CONTRIBUTING.md conformance: title is valid Conventional Commits, the PR body keeps the template headings, the changeset is present (minor, appropriate for a new export), the spec is updated, and tests are colocated. The unchecked catalog item has a reason. I haven't run the local gate (CI is still in progress), so I'm relying on the stated results.

Suggestions (non-blocking):

  1. packages/activitypub/src/delivery.ts: KNOCK_AGAIN_STATUSES includes 403. A peer that blocks our domain or actor also answers 403, so every delivery to that peer is sent twice, and the second knock will also fail. The fallback is probably still worth it for peers that use 403 for signature failures. Consider a short comment saying the doubled traffic to blocking peers is accepted, or record the failure so the next knock is skipped.
  2. delivery.ts: deliverActivity now takes 7 positional parameters, with preference added after timeoutMs. Callers that only want to pass a preference must also supply timeoutMs. Moving the optional trailing arguments into an options object would be cleaner. This is non-blocking, but a public signature is easier to change before release.
  3. packages/activitypub/src/object.ts: the import type { SignatureProfile } from "@dwk/http-signatures" sits between the ./as2.js import and the @dwk/log import. Move it to the other package imports for consistency.
  4. object.ts: sig_pref rows are only written when the second knock succeeds and preference !== "cavage". After the 7-day TTL expires, a draft-cavage-only peer costs one wasted RFC 9421 knock per actor DO, which is acceptable. A test for the expiry path (a row older than 7 days is ignored and pruned) would lock that behaviour in, if one doesn't already exist.

No blocking issues found. Please wait for CI to go green and resolve any threads before merging.


Generated by Claude Code

Move deliverActivity's optional trailing arguments into a
DeliveryOptions object, document the accepted 403 double-traffic
trade-off, regroup the type import, and test the sig_pref TTL expiry
and prune path.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_015LRGjmHE99WESN5HBERxSj
Open a host's delivery circuit after 5 consecutive retryable failures
and defer its rows (no retry attempt spent) for a cooldown that starts
at 10 minutes and doubles per failed half-open probe, capped at 24 h.
A success, a non-retryable answer, or a signed inbound activity from
the host closes it; after 7 days of failure, rows reaching an open
circuit are dropped. State lives in the actor DO's SQLite.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_015LRGjmHE99WESN5HBERxSj
@davidwkeith davidwkeith changed the title feat(activitypub): double-knock outbound delivery signatures feat(activitypub): double-knock delivery signatures and per-host circuit breaker Oct 28, 2026
@davidwkeith
davidwkeith merged commit d71578a into main Oct 28, 2026
9 checks passed
@davidwkeith
davidwkeith deleted the claude/trusting-hawking-3eawag branch October 28, 2026 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants