Skip to content

fix(mcp): stabilize authenticated tool discovery#5754

Open
j15z wants to merge 25 commits into
stagingfrom
fix/mcp-discovery-races
Open

fix(mcp): stabilize authenticated tool discovery#5754
j15z wants to merge 25 commits into
stagingfrom
fix/mcp-discovery-races

Conversation

@j15z

@j15z j15z commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

Authenticated MCP servers that advertise OAuth can now be added and refreshed with an explicitly configured static Authorization: Bearer ... credential without being diverted into OAuth. The previous fix made the pre-save connection test try the configured header, but runtime discovery still mapped every SDK UnauthorizedError to “authorization required.” This PR makes that classification depend on the server's configured auth type, so static-header servers stay on the normal credential path and only OAuth-configured servers enter OAuth.

OAuth-pending connections now remain actionable in Workspace Settings: the persistent server status says OAuth authorization required, and a refresh that cleanly returns to that state says Authorization required instead of the generic Failed / Not Connected. Concrete connection errors still take precedence, so timeouts and other failures are not mislabeled as an OAuth prompt.

Discovery publication is ordered end to end. Redis and memory cache adapters allocate a monotonic millisecond mutation token; conditional cache publication and the database lastToolsRefresh compare-and-swap use that same token, so retries, failures, successes, and invalidations choose one winner across both layers. A credential-safe discovery revision covers every connection-affecting field, preserving valid tools across metadata-only edits while failing closed after URL, credential, auth-type, or transport changes.

Each retry acquires fresh mutation ownership. Successful, failed, and OAuth-pending status publication share a bounded retry for metadata-only edit races: every attempt rechecks cache ownership and the discovery revision before publishing against the latest row token. If ordering or an atomic cache transition is unavailable, freshly fetched tools may be returned to the current caller but unordered cache/database state is not persisted. Successful discovery returns its exact publication token, so workflow schemas sync only while the post-discovery reread still belongs to that winning published refresh; later success, failure, or invalidation barriers take precedence, and winner-cache, superseded, and degraded unavailable results cannot propagate as winners. Cache invalidation retries reacquire fresh ownership, publish a database barrier only after an atomic delete wins, and never fall back to an unordered delete that could erase a newer discovery.

Refresh responses now prove that a newer successful discovery actually won before preserving connected state after a failure: both lastToolsRefresh and lastConnected must advance beyond the request's original snapshot. Invalidation advances the publication token but not lastConnected, so it can no longer hide a real discovery failure behind a stale connected row. Failure counters also recompute through an optimistic JSONB compare-and-swap when success races them.

Security-sensitive paths remain bounded: MCP targets still pass domain and SSRF validation, public targets are DNS-pinned when policy permits, and HTTP/2 negotiation does not relax that pinning. Logs contain header names and structural diagnostics but never values, session IDs, or raw upstream bodies, while UI/API errors come from an allowlist. OAuth servers without dynamic client registration return an actionable 422 that directs users to configure a token. The settings copy change consumes only the existing auth type and sanitized error state; it does not alter or expose credentials, headers, tokens, or upstream responses.

Related: #5665 and #5595.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Other: ___________

Testing

  • bun run --cwd apps/sim test -- lib/mcp app/api/mcp 'app/workspace/[workspaceId]/settings/components/mcp' — 454 tests passed across all 29 MCP suites, including static Bearer classification, concurrent publication, repeated metadata-only races, ownership-safe invalidation retries, degraded-cache behavior, publication-token-gated workflow sync, secret-safe diagnostics, SSRF pinning, invalidation-only refresh races, and OAuth-specific settings status behavior.
  • bun run check:api-validation:strict — passed with all 964 routes Zod-backed and no boundary-policy drift.
  • Biome on the final review files and git diff --check — passed.
  • Direct GitHub Copilot MCP probes completed initialize and tools/list with 47 tools using both normal and SSRF-pinned transports; the SDK transport carried the session ID without logging it.
  • A full app-suite baseline recorded 10,715 passing tests and 41 known failures in unrelated workflow, webhook, chat, and UI tests. The final review changes were revalidated in the focused suite above.
  • bun run --cwd apps/sim type-check reaches only unrelated shared-worktree dependency errors in fork audit events, recordAuditBatch, and @google-cloud/storage; no MCP TypeScript errors are reported.

Reviewer focus: auth-type-aware handling of typed 401s, OAuth-pending UI status with concrete-error precedence, unified cache/database mutation ownership, bounded same-revision status retries, fresh-token invalidation retries, Redis conditional-mutation scripts, winner-only workflow synchronization, fail-closed degraded-cache behavior, and the two-timestamp proof for a racing successful refresh.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Screenshots/Videos

N/A — status-copy-only UI change with no layout change.

@vercel

vercel Bot commented Jul 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 18, 2026 11:28am

Request Review

@cursor

cursor Bot commented Jul 18, 2026

Copy link
Copy Markdown

PR Summary

High Risk
Touches OAuth classification, concurrent discovery publication, and SSRF-pinned transport behavior—areas where races or misclassification could affect connection state and security-sensitive outbound calls.

Overview
This PR tightens MCP auth and discovery so static-header servers are not treated like OAuth when the SDK returns UnauthorizedError, while OAuth-pending servers get clearer settings copy (OAuth authorization required / refresh Authorization required when there is no concrete error). OAuth start now returns 422 with guidance when dynamic client registration is unsupported.

Tool discovery and refresh are reworked around a shared monotonic cache mutation id (memory + Redis) that also drives database lastToolsRefresh compare-and-swap, plus a discovery revision hash so metadata-only edits do not invalidate tools. The refresh API uses discoverServerToolsWithMetadata to handle published, superseded, degraded-cache, and OAuth-pending outcomes—skipping workflow sync when a newer refresh or invalidation wins—and no longer bumps refresh timestamps or clears workspace cache on every success.

Supporting changes: HTTP/2 on pinned MCP fetch, redacted structural error diagnostics in logs, allowlisted user-facing failure messages, and workspace cache invalidation when auth type or transport changes on server update.

Reviewed by Cursor Bugbot for commit b9dfe48. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread apps/sim/lib/mcp/service.ts Outdated
Comment thread apps/sim/lib/mcp/client.ts
@greptile-apps

greptile-apps Bot commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR stabilizes authenticated MCP tool discovery and publication. The main changes are:

  • Classifies unauthorized responses according to the configured authentication type.
  • Keeps OAuth authorization states actionable in the API and workspace settings.
  • Orders cache and database updates with shared mutation tokens.
  • Uses configuration revisions and timestamp bounds to prevent stale publication.
  • Preserves SSRF pinning while enabling HTTP/2 for MCP traffic.
  • Sanitizes connection diagnostics and OAuth errors.

Confidence Score: 5/5

This looks safe to merge.

  • The timestamp range includes PostgreSQL sub-millisecond values while excluding the next millisecond.
  • Ordered mutation checks prevent older discoveries from overwriting newer cache or database state.
  • No blocking issues were found in the updated publication path.

Important Files Changed

Filename Overview
apps/sim/lib/mcp/service.ts Coordinates discovery publication across cache and database state, including timestamp precision, retries, invalidation, and superseded refreshes.
apps/sim/lib/mcp/storage/redis-cache.ts Adds atomic Redis mutation ownership and conditional cache updates.
apps/sim/lib/mcp/storage/memory-cache.ts Adds ordered mutation ownership and conditional updates to the memory cache.
apps/sim/app/api/mcp/servers/[id]/refresh/route.ts Uses discovery metadata and persisted timestamps to report refresh outcomes without preserving stale connected state.
apps/sim/lib/mcp/client.ts Makes unauthorized handling auth-type-aware and limits logged error details to sanitized structural fields.

Reviews (4): Last reviewed commit: "Address PR review feedback (#5754)" | Re-trigger Greptile

Comment thread apps/sim/lib/mcp/service.ts Outdated
j15z added a commit that referenced this pull request Jul 18, 2026
- preserve typed static-header authentication failures\n- order discovery publication by start time\n- atomically publish tools and failure cache state
Comment thread apps/sim/lib/mcp/service.ts Outdated
Comment thread apps/sim/lib/mcp/service.ts
j15z added a commit that referenced this pull request Jul 18, 2026
Match configuration generations within the JavaScript millisecond window so PostgreSQL microseconds do not suppress valid publication.
@j15z

j15z commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator Author

The database publication predicate needs a precision-safe configuration generation.

Addressed in 89dce25: both status-publication predicates now match the JavaScript millisecond window, accepting PostgreSQL sub-millisecond precision while still rejecting the next generation. A boundary regression covers the behavior.

Comment thread apps/sim/lib/mcp/service.ts
j15z added a commit that referenced this pull request Jul 18, 2026
- fall back to best-effort cache writes when mutation ordering is unavailable\n- share bounded redacted MCP error diagnostics across client and service
Comment thread apps/sim/lib/mcp/service.ts Outdated
j15z added a commit that referenced this pull request Jul 18, 2026
Retry cache mutation ownership before publishing discovery state, avoid unordered publication that older writers can supersede, and keep explicit invalidation best effort.
Comment thread apps/sim/lib/mcp/service.ts
Comment thread apps/sim/lib/mcp/storage/redis-cache.ts
j15z added a commit that referenced this pull request Jul 18, 2026
Use timestamp-based mutation tokens for both cache ownership and database CAS ordering, and invalidate Redis mutation owners before deleting entries during a full clear.
Comment thread apps/sim/lib/mcp/service.ts Outdated
Comment thread apps/sim/lib/mcp/service.ts Outdated
j15z added a commit that referenced this pull request Jul 18, 2026
Recompute consecutive failures after status CAS conflicts, reload winning cached tools for superseded discoveries, and return live tools without unordered publication when cache ordering is unavailable.
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts
j15z added a commit that referenced this pull request Jul 18, 2026
Expose discovery publication metadata to refresh callers, report live tools during cache degradation, and compare lastToolsRefresh mutation tokens instead of wall clocks when preserving a newer success.
Comment thread apps/sim/lib/mcp/service.ts Outdated
j15z added a commit that referenced this pull request Jul 18, 2026
Probe mutation ownership after database CAS misses, keep valid live tools for metadata-only edits, and invalidate ownership for transport and auth-type changes.
Comment thread apps/sim/lib/mcp/service.ts Outdated
Comment thread apps/sim/lib/mcp/orchestration/server-lifecycle.ts Outdated
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts
- Clear MCP caches when implicit OAuth promotion changes the resolved auth type
- Cover unchanged-client-ID promotion with a lifecycle regression test
Comment thread apps/sim/lib/mcp/service.ts Outdated
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts
- Fall back safely when ordered MCP cache invalidation is unavailable
- Surface sanitized concrete refresh errors without masking OAuth-pending state
@j15z

j15z commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile

Comment thread apps/sim/lib/mcp/service.ts
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts
- Retry failure and OAuth status publication across metadata-only races
- Keep losing winner-cache refreshes from syncing workflows
Comment thread apps/sim/lib/mcp/service.ts Outdated
- Retry successful status publication across metadata-only races
- Preserve superseded and unavailable discovery semantics
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts
- Preserve OAuth-pending state when a status reread is stale
- Keep concrete refresh errors sanitized and visible
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts
Comment thread apps/sim/lib/mcp/service.ts Outdated
- Make cache invalidation retries ownership-safe
- Preserve proven newer successes for superseded refreshes
Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 0fb9a4e. Configure here.

Comment thread apps/sim/app/api/mcp/servers/[id]/refresh/route.ts
@j15z
j15z requested a review from waleedlatif1 July 18, 2026 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants