Skip to content

fix(grid-wallet-prod): replay missed webhooks to reconnecting SSE clients#742

Open
pengying wants to merge 2 commits into
07-21-feat_grid-wallet-prod_fork_grid-wallet-demo_as_the_production_demo_appfrom
07-24-fix_grid-wallet-prod_replay_missed_webhooks_to_reconnecting_sse_clients
Open

fix(grid-wallet-prod): replay missed webhooks to reconnecting SSE clients#742
pengying wants to merge 2 commits into
07-21-feat_grid-wallet-prod_fork_grid-wallet-demo_as_the_production_demo_appfrom
07-24-fix_grid-wallet-prod_replay_missed_webhooks_to_reconnecting_sse_clients

Conversation

@pengying

@pengying pengying commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Summary

Fan-out to multiple clients already worked; replay on reconnect didn't. This adds it.

The demo drives one hardcoded customer, so every connected panel is watching the same account and should see every delivery. There is no per-client filtering — pushEvent walks all subscribers — and that part is verified below. The gap was that a client only ever saw events that arrived while it was connected, so an EventSource auto-reconnect after a network blip silently dropped whatever landed in the gap.

What changed

  • Every frame now carries an id: (a per-process sequence number). EventSource echoes the last one back as Last-Event-ID when it reconnects, and the stream replays events after that point.
  • A fresh client replays nothing. It presents no id, and replaying history would render old deliveries as if they had just arrived — the panel timestamps entries on receipt.
  • Subscribe happens before replay, so a delivery landing mid-replay queues behind it rather than slipping between the two steps.
  • The connect handshake comment reports clients=N. EventSource ignores comment frames; it shows up in curl -N .../api/webhooks/stream when you want to confirm every panel is attached.

Verified

Three concurrent clients, one delivery each time, with a disconnect and a reconnect in the middle:

A handshake: : connected clients=1     B: clients=2     C: clients=3
deliver #1 → A [1] | B [1] | C [1]                    ← every client, no filtering

B disconnects; #2 and #3 land while it is away
  A [1,2,3] | C [1,2,3]                               ← survivors unaffected
B reconnects with Last-Event-ID: 1
  B replayed: [2,3]                                   ← exactly what it missed
D connects fresh: []                                  ← no invented history
deliver #4 → A [1,2,3,4] | C [1,2,3,4] | B [2,3,4] | D [4]

Signature verification is untouched; the replay path only re-sends events that already passed it.

Limits (documented in the module)

  • The bus is per server process. With multiple replicas, only the instance that received a delivery can fan it out to its own clients — a shared channel (Redis pub/sub) would be needed.
  • The ring buffer holds the last 50 events, so a client away longer than that loses the overflow.

Stacked on #730.

Update: catch-up must not duplicate either

Self-review caught a flaw in the first push. Subscribe-then-replay avoids gaps but double-sends: an event landing between subscribe and the replay loop goes out live and again from the backlog. Replay-then-subscribe has the opposite bug (it loses that event). The stream now subscribes immediately but holds live events, replays the backlog, then flushes the held ones the replay didn't cover.

Verified by reconnecting from Last-Event-ID: 1 while deliveries stream in:

client saw: 2,3,4,5,6
duplicates: none
gaps      : none (2..6 as expected)

Also added webhookEvents.test.ts covering the bus contract: fan-out to every subscriber, survival of a subscriber that throws, the seq/eventsSince replay windows, and the ring dropping the overflow a long-gone client can no longer catch up on. 69 tests total (was 60).

Update: the stream was readable by any site

Second self-review finding, and a more serious one. /api/webhooks/stream is unauthenticated and carries whatever Grid delivered — transaction ids, amounts, and counterpartyInformation, which per Grid's schema can include a counterparty's name, birth date and nationality. That is the same exposure GET /api/webhooks/events was deleted for earlier in this stack; the SSE replacement inherited it.

An EventSource opened by another site is now refused (Sec-Fetch-Site: cross-site → 403), so a page on someone else's origin can't attach to a panel running on your machine:

same-origin (our own panel)        → 200
no header (curl / scripts)         → 200
cross-site (another website)       → 403

This is not access control, and the code comment says so: anyone who can reach the host can still read the stream, which behind an ngrok tunnel means anyone with the URL. Gating it properly — a session cookie, or not exposing it past localhost — belongs on the pre-prod list next to proxy customer-scoping.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Pjqydq34z8DBK1kBrioXK2

@vercel

vercel Bot commented Jul 25, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Actions Updated (UTC)
grid-flow-builder Ignored Ignored Preview Jul 25, 2026 3:57am
grid-wallet-demo Ignored Ignored Preview Jul 25, 2026 3:57am

Request Review

pengying commented Jul 25, 2026

Copy link
Copy Markdown
Contributor Author

…ents

Every connected panel already received every delivery — the demo drives one
hardcoded customer, so there is no per-client filtering and pushEvent walks all
subscribers. Verified with three concurrent clients: all three got the event, and
one disconnecting left the others streaming.

What was missing is replay. A client only saw events that arrived while it was
connected, so an EventSource auto-reconnect after a network blip silently dropped
anything that landed in the gap.

Each frame now carries an `id:` (a per-process sequence). EventSource echoes it
back as `Last-Event-ID` on reconnect, and the stream replays events after that
point. A FRESH client replays nothing: it has no id, and rendering old deliveries
would date them to the moment of receipt.

Catch-up must neither gap nor duplicate, which rules out both naive orderings —
replay-then-subscribe loses whatever lands in between, and subscribe-then-replay
sends that same event twice (live, then again from the backlog). So the stream
subscribes immediately but HOLDS live events, replays the backlog, then flushes
the held ones the replay didn't already cover. Verified by reconnecting while
deliveries stream in: no gaps, no duplicates.

The connect handshake comment now reports `clients=N` — invisible to EventSource,
visible in `curl -N`, useful when checking that every panel is attached.

Tests cover the bus contract: fan-out to every subscriber, survival of a broken
one, the seq/eventsSince replay windows, and the ring dropping the overflow a
long-gone client can no longer catch up on.

Limits unchanged and documented in the module: the bus is per server process (with
replicas, only the instance that received a delivery fans it out), and the ring
buffer holds the last 50 events.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01Pjqydq34z8DBK1kBrioXK2
@pengying
pengying force-pushed the 07-24-fix_grid-wallet-prod_replay_missed_webhooks_to_reconnecting_sse_clients branch from d930b99 to d15af0e Compare July 25, 2026 03:54
… stream

Self-review follow-up. /api/webhooks/stream is unauthenticated and carries whatever
Grid delivered — transaction ids, amounts, and counterpartyInformation, which per
Grid's schema can include a counterparty's name, birth date and nationality. That is
the same exposure the polling route was deleted for earlier in this stack; the SSE
replacement inherited it.

Reject an EventSource opened by another site (Sec-Fetch-Site: cross-site), so a page
on someone else's origin can't attach to a panel running on your machine.
Deliberately narrow: only an explicit cross-site is refused, so `curl -N` and the
verification scripts keep working.

This is NOT access control, and the comment says so: anyone who can reach the host
can still read the stream, which behind a tunnel means anyone with the URL. Gating
it properly (session cookie, or not exposing it past localhost) stays a pre-prod
requirement alongside proxy customer-scoping.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01Pjqydq34z8DBK1kBrioXK2
@pengying

Copy link
Copy Markdown
Contributor Author

@greptileai review

Context for the review: this is a small SSE change with two self-review fixes already applied — reconnect catch-up (hold-and-flush so it neither gaps nor duplicates) and a cross-site subscription refusal. The parts I'd most like a second pair of eyes on:

  1. src/app/api/webhooks/stream/route.ts — the hold/replay/flush ordering, and cleanup on abort (unsubscribe + clearInterval + controller.close).
  2. src/lib/webhookEvents.ts — the bus is pinned to globalThis because Next gives each route handler its own module instance; is that the right call here?
  3. Whether refusing only Sec-Fetch-Site: cross-site is the right narrowness, given the stream is otherwise unauthenticated (documented as a pre-prod gap).

@greptile-apps

greptile-apps Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds SSE reconnect replay to the grid-wallet-prod demo: each event frame now carries an id: sequence number, reconnecting clients send Last-Event-ID, and the server replays whatever they missed from a 50-event ring buffer. A second self-review fix addresses the subscribe-then-replay gap/duplicate race with a hold-and-flush mechanism, and a third adds a Sec-Fetch-Site: cross-site guard to block cross-origin EventSource subscriptions.

  • Hold-and-flush replay: on reconnect, the handler subscribes first (buffering live arrivals in held), replays the ring-buffer backlog, then flushes only held events not already covered by replay — eliminating both gaps and duplicates.
  • globalThis bus: the webhook bus is pinned to globalThis instead of module scope so the publisher route and stream route share the same registry across Next.js's per-route module instances.
  • Cross-site guard: Sec-Fetch-Site: cross-site → 403; curl, same-origin, and same-site requests all pass through (explicitly documented as a pre-prod auth gap).

Confidence Score: 5/5

Safe to merge — the reconnect logic, ring-buffer replay, and resource cleanup are all correct, and the cross-site guard is intentionally narrow and well-documented.

The hold-and-flush ordering is sound: subscribing before replay and gating live events through held correctly prevents both gaps and duplicates. Cleanup is covered by two complementary idempotent paths (abort handler + cancel()), and Set.delete plus a guarded clearInterval make double-invocation harmless. The globalThis bus pin is the right call for Next.js's per-route module isolation. No data loss or resource-leak paths were found.

Files Needing Attention: No files require special attention. The Sec-Fetch-Site guard and its documented limitations in route.ts are worth a pre-prod revisit, but that is explicitly tracked and does not affect correctness of the replay feature.

Important Files Changed

Filename Overview
components/grid-wallet-prod/src/app/api/webhooks/stream/route.ts Adds reconnect-replay with hold-and-flush ordering, cross-site guard, and id: frame emission; cleanup on abort is idempotent across both the abort handler and cancel() paths.
components/grid-wallet-prod/src/lib/webhookEvents.ts Adds seq counter, eventsSince(), and listenerCount() to the globalThis-pinned bus; ring-buffer trimming and fan-out error isolation unchanged.
components/grid-wallet-prod/src/lib/webhookEvents.test.ts New test suite covering fan-out, broken-subscriber survival, listener count, seq ordering, eventsSince windows, and ring-buffer overflow; clearEvents() not resetting seq is correct and the tests account for it.

Sequence Diagram

sequenceDiagram
    participant C as Client (EventSource)
    participant R as route.ts GET
    participant B as webhookEvents bus

    Note over C,B: Initial connection (no Last-Event-ID)
    C->>R: GET /api/webhooks/stream
    R->>R: crossSite check pass
    R->>B: "subscribe holding=true"
    R->>C: ": connected clients=1"
    R->>R: "isFinite(NaN)=false, skip replay"
    R->>R: "holding=false, flush held (empty)"
    R->>C: keepalive every 15s

    Note over C,B: Event arrives while connected
    B->>R: pushEvent fires listener
    R->>C: "id: 1 data: {...}"

    Note over C,B: Network blip, client disconnects
    C--xR: connection lost

    Note over C,B: Auto-reconnect with Last-Event-ID: 1
    C->>R: GET (Last-Event-ID: 1)
    R->>R: crossSite check pass
    R->>B: "subscribe holding=true"
    Note over R: concurrent events go to held[]
    R->>B: eventsSince(1) returns [2,3]
    R->>C: "id: 2 data: {...}"
    R->>C: "id: 3 data: {...}"
    R->>R: "replayedThrough=3, holding=false"
    R->>R: "flush held where seq > 3"

    Note over C,B: Client closes tab
    C->>R: abort signal fires
    R->>B: unsubscribe()
    R->>R: clearInterval(keepalive)
    R->>R: controller.close()
Loading

Reviews (2): Last reviewed commit: "fix(grid-wallet-prod): refuse cross-site..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant