Skip to content

OUT-4104: Skip the sync job when a Dropbox webhook has no relevant changes - #143

Closed
SandipBajracharya wants to merge 5 commits into
OUT-4101from
OUT-4104
Closed

OUT-4104: Skip the sync job when a Dropbox webhook has no relevant changes#143
SandipBajracharya wants to merge 5 commits into
OUT-4101from
OUT-4104

Conversation

@SandipBajracharya

Copy link
Copy Markdown
Collaborator

Stacked on OUT-4101 (PR #142) — review/merge that first.

Dropbox fires a webhook for any change in the account, including changes outside the synced folders. Today every non-debounced webhook starts a processDropboxChanges Trigger.dev job that lists, filters to empty, and does nothing. This adds a cheap pre-check so the job only starts when there's actually something to sync.

What changed (webhook.service.ts)

  • handleDropboxEvents (non-debounced branch) now calls triggerIfPendingChanges instead of always triggering.
  • accountHasPendingChanges(account) — read-only peek: for each active channel, filesListFolderContinue(storedCursor) and short-circuit true on the first entry (added or deleted) under the channel's dbxRootPath (via a stack-safe recursive deltaHasRelevantEntry). It never persists the advanced cursor — the job re-fetches from the stored one.
  • Skips the job when nothing relevant changed (change outside synced roots, or account with no mapped channels).

Fail-open (never drop a real change)

Every uncertain path triggers the job: unreadable connection → trigger; channel with no cursor → trigger; missing path_display (unmounted/edge entries) → treated as relevant → trigger; any thrown error (auth, Dropbox 409/reset) → caught and triggered. The root match is anchored (path === root || startsWith(root + '/')) so /root doesn't match a sibling /rootbar.

Testing

New dropbox-webhook-precheck.integration.test.ts: skip when delta is all outside root; trigger when under root; trigger on null cursor; skip when no channels; trigger on missing path_display; not match a sibling prefix; throws surfaced to the caller; and fail-open trigger through handleDropboxEvents when the pre-check throws. One debounce test updated to isolate timing from the pre-check.

  • 273 unit + 187 integration pass; pnpm typecheck + pnpm lint clean.

Scope / follow-up

  • Direct-trigger path only. The debounce→cron catch-up path still triggers directly (bypassing the pre-check). Applying the same check there is a tracked follow-up (reuses accountHasPendingChanges).
  • Cost note: for a noisy account whose changes are all irrelevant, the pre-check runs per webhook rather than engaging the 5-min debounce (since the job — which stamps lastWebhookSyncStartedAt — never runs). Trade is cheap short-circuited peeks vs. avoided Trigger.dev runs; the cron follow-up (with stamp-on-check) is where this gets fully bounded.

🤖 Generated with Claude Code

@linear-code

linear-code Bot commented Aug 26, 2026

Copy link
Copy Markdown

OUT-4104

@vercel

vercel Bot commented Aug 26, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
dropbox-integration Ready Ready Preview Aug 27, 2026 9:38am

Request Review

@greptile-apps

greptile-apps Bot commented Aug 26, 2026

Copy link
Copy Markdown

Greptile Summary

The PR adds a Dropbox delta pre-check to avoid unnecessary sync jobs and moves webhook processing behind an immediate acknowledgement. It also introduces per-account failure isolation and pending-webhook recovery, but the delivery is still not durably owned before Dropbox receives success.

  • Defers account processing through Next.js after() and extends the route duration.
  • Scans channel cursors for relevant Dropbox paths before triggering synchronization.
  • Adds per-account failure handling and integration coverage for pre-check, debounce, and route behavior.

Confidence Score: 4/5

The PR is not yet safe to merge because Dropbox can receive a successful acknowledgement before the notification has any durable task or recovery state.

The deferred callback can be terminated before ownership is recorded, and a transient connection-query failure is swallowed before the account can be marked pending, leaving the catch-up schedule unable to recover the acknowledged change.

Files Needing Attention: src/features/webhook/dropbox/api/webhook.controller.ts and src/features/webhook/dropbox/lib/webhook.service.ts

Important Files Changed

Filename Overview
src/features/webhook/dropbox/api/webhook.controller.ts Defers webhook work until after acknowledgement without first creating a durable handoff, allowing an acknowledged delivery to be lost.
src/features/webhook/dropbox/lib/webhook.service.ts Adds the delta pre-check and per-account recovery, but connection-query failures remain outside the pending-state safety path.
src/app/api/webhook/dropbox/route.ts Extends the route duration to accommodate deferred work, while still leaving that work bounded by the request runtime.
test/flows/dropbox-webhook-precheck.integration.test.ts Covers relevant-path detection, fail-open behavior, null cursors, and missing Dropbox paths.
test/flows/dropbox-webhook-debounce.integration.test.ts Covers debounce recovery and account isolation but not failures before a connection ID is obtained.
test/flows/dropbox-webhook-route.integration.test.ts Verifies immediate acknowledgement and swallowed callback errors, but its captured callback does not exercise runtime termination or durable delivery ownership.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  D[Dropbox webhook] --> V[Validate signature]
  V --> A[Return 200 and register after callback]
  A --> P[Process accounts]
  P --> C{Relevant changes?}
  C -->|Yes or uncertain| T[Queue sync task]
  C -->|No| S[Skip task]
  P -->|Caught account failure| M[Mark connection pending]
  M --> R[Scheduled catch-up]
  A -. callback termination or pre-ID lookup failure .-> L[No task and no pending state]
Loading

Reviews (3): Last reviewed commit: "fix(OUT-4104): recover failed webhook ac..." | Re-trigger Greptile

Comment thread src/features/webhook/dropbox/lib/webhook.service.ts
@SandipBajracharya

Copy link
Copy Markdown
Collaborator Author

@greptileai review again

Comment on lines +51 to +59
after(async () => {
try {
await sleep(800) // let our own writes settle first
await new DropboxWebhook().handleDropboxEvents(accounts)
} catch (error) {
// Dropbox already got its 200, so it won't retry — report so this is visible.
console.error('Dropbox webhook :: background processing failed', { accounts }, error)
Sentry.captureException(error)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Deferred delivery is lost

When deferred account processing encounters a database, Dropbox, or Trigger.dev failure, the callback swallows the error after the route has returned 200. Because neither a sync task nor pendingWebhook is guaranteed to exist at that point, Dropbox does not retry and the affected account—plus any later accounts in the sequential loop—can remain unsynchronized.

Knowledge Base Used:

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Each account is now processed in its own try/catch inside the loop. If it fails, we mark that connection's pendingWebhook flag so the catch-up cron re-runs it, and keep going with the other accounts — so a failure is recovered instead of silently dropped, and one bad account no longer blocks the rest. The flag is scoped to the specific connection row. Added tests for both (a failed account gets marked, and a later account still runs).

@SandipBajracharya

Copy link
Copy Markdown
Collaborator Author

@greptileai review again

const dropboxWebhook = new DropboxWebhook()
await dropboxWebhook.handleDropboxEvents(accounts)
// Reply to Dropbox first, then process in the background so the check doesn't slow the reply.
after(async () => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Deferred delivery lacks durability

When the after() callback is terminated before processing an account, or the active-connection lookup fails, Dropbox has already received 200 but no sync task or pending state exists. The catch-up schedule only selects connections successfully marked pending, so the acknowledged change remains unsynchronized unless another webhook later arrives.

Knowledge Base Used:

@SandipBajracharya

Copy link
Copy Markdown
Collaborator Author

Closing unmerged — OUT-4104 is cancelled. Doing the change-check in the web app to skip no-op runs meant giving up Dropbox + Trigger.dev retry durability, which kept surfacing dropped-change risks; the benefit over the existing debounce was marginal. Letting the sync task do the check (its current behaviour) is simpler and durable. Nothing to revert on the mainline since this only lived on this branch. If no-op run cost proves significant (OUT-4102), revisit with a durable webhook inbox rather than a web-app pre-check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant