Skip to content

feat(mcp): add batch stream config overrides and update_cloud_connection_unsafe - #1397

Open
Aaron ("AJ") Steers (aaronsteers) wants to merge 17 commits into
release-candidate/v1from
devin/1792179543-rc-stream-sync-mode
Open

Aaron ("AJ") Steers (aaronsteers) wants to merge 17 commits into
release-candidate/v1from
devin/1792179543-rc-stream-sync-mode

Conversation

@aaronsteers

@aaronsteers Aaron ("AJ") Steers (aaronsteers) commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Overview

👉 TL;DR: Agents can change how several streams on a Cloud connection sync, all in one call. The default tool refuses changes that would invalidate a stream's saved progress, and a separate unsafe tool allows them.

Specifically, this adds override_stream_configs: list[StreamConfigOverride] to update_cloud_connection (now destructive=False), the new update_cloud_connection_unsafe (destructive=True), and CloudConnection.update_stream_configs.

Merges before (pairs with) #1392:

Replaces (with #1392):

Changes

  • StreamConfigOverride: stream_name, stream_namespace (None = any, "" = unnamespaced), sync_mode, destination_sync_mode, cursor_field (top-level str). At least one setting is required.
  • CloudConnection.update_stream_configs(overrides, *, allow_unsafe=False): the only public entry point. In one call it reads the catalog and state, evaluates every override on a typed ConfiguredAirbyteCatalog (private _evaluate_stream_config_updates; API-only fields like selected/mappers round-trip unchanged), and writes the catalog once. Helpers live in private airbyte/cloud/_stream_config_overrides.py. No method takes a precomputed plan. Returns StreamConfigUpdateResult; reset_streams is only filled when a reset was requested.
  • Validation errors (AirbyteLibInputError): missing stream (the available streams are listed once per call), ambiguous name, unsupported sync mode, incremental with no cursor, and *_dedup with no primary key. Any invalid override rejects the whole batch.
  • Safety check: _find_unsafe_stream_changes flags changes to syncMode, destinationSyncMode, cursorField, or primaryKey on streams with saved state, and the safe path raises AirbyteUnsafeStreamConfigChangeError (naming each stream and changed field) without writing anything.
  • MCP update_cloud_connection: gains override_stream_configs, drops to destructive=False, always sends skipReset=true, and writes stream overrides before enabled/schedule.
  • New MCP update_cloud_connection_unsafe: same inputs; sends skipReset=false when unsafe streams exist and lists every stream the platform will refresh or reset. The MCP layer only adds error guidance; arg validation and schedule setting live in private airbyte/cloud/_connection_settings.py.
  • api_util.replace_connection_catalog and import_raw_catalog take skip_reset (default True).
  • import_raw_catalog clears the cached _connection_info (shared with feat(mcp): add async source catalog discovery and comparison (discover_cloud_source_catalog) #1392).

Review Spotlight

Reviewers with limited time, please review first:

Risks

Test plan

  • AIRBYTE_MCP_TRUSTED_EXECUTION=1 uv run pytest tests/unit_tests/test_cloud_connection_catalog_edits.py tests/unit_tests/test_mcp_connection_schedules.py tests/unit_tests/test_mcp_tool_errors.py tests/unit_tests/test_mcp_arg_trace_parity.py
  • ruff check, ruff format --check

Link to Devin session: https://app.devin.ai/sessions/4d1ece6885c44c52a07034e025db11e6
Open in Devin Desktop: https://app.devin.ai/desktop/session/4d1ece6885c44c52a07034e025db11e6?variant=devin
Requested by: Aaron ("AJ") Steers (@aaronsteers)

@devin-ai-integration

Copy link
Copy Markdown
Contributor

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@github-actions

Copy link
Copy Markdown

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This PyAirbyte Version

You can test this version of PyAirbyte using the following:

# Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1792179543-rc-stream-sync-mode' pyairbyte --help

# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1792179543-rc-stream-sync-mode'

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /fix-pr - Fixes most formatting and linting issues
  • /uv-lock - Updates uv.lock file
  • /test-pr - Runs tests with the updated PyAirbyte
  • /prerelease - Builds and publishes a prerelease version to PyPI
📚 Show Repo Guidance

Helpful Resources

Community Support

Questions? Join the #pyairbyte channel in our Slack workspace.

📝 Edit this welcome message.

@github-code-quality

github-code-quality Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Code coverage overview

Languages: Python

Python / code-coverage/pytest-no-creds

release-candidate/v1 d9f652f devin/1792179543-rc-... 585268d +/-
65% 80% +15%
View coverage of the 10 most impacted files.
File release-candidate/v1 d9f652f devin/1792179543-rc-... 585268d +/-
airbyte/_util/api_util.py 36% 67% +31%
airbyte/mcp/cloud.py 52% 85% +33%
airbyte/cloud/connectors.py 34% 79% +45%
airbyte/mcp/_datadog.py 0% 50% +50%
airbyte/mcp/_telemetry.py 0% 94% +94%
airbyte/cloud/c...lder/project.py 0% 94% +94%
airbyte/_direct...ctors/models.py 0% 95% +95%
airbyte/mcp/_sso_auth.py 0% 96% +96%
airbyte/cloud/models.py 0% 97% +97%
airbyte/mcp/int...nc_status_ui.py 0% 97% +97%

Python / code-coverage/pytest

release-candidate/v1 d9f652f devin/1792179543-rc-... 585268d +/-
71% 82% +11%
View coverage of the 10 most impacted files.
File release-candidate/v1 d9f652f devin/1792179543-rc-... 585268d +/-
airbyte/cloud/connectors.py 51% 82% +31%
airbyte/mcp/cloud.py 52% 85% +33%
airbyte/mcp/_datadog.py 0% 50% +50%
airbyte/mcp/_user_identity.py 0% 93% +93%
airbyte/mcp/_telemetry.py 0% 94% +94%
airbyte/cloud/c...lder/project.py 0% 94% +94%
airbyte/_direct...ctors/models.py 0% 95% +95%
airbyte/mcp/_sso_auth.py 0% 96% +96%
airbyte/cloud/models.py 0% 97% +97%
airbyte/mcp/int...nc_status_ui.py 0% 97% +97%

Python / code-coverage/pytest-unit

release-candidate/v1 devin/1792179543-rc-... 585268d +/-
— 77% —

Line coverage data for the release-candidate/v1 branch is not yet available.

View coverage of the 10 most covered files.
File release-candidate/v1 devin/1792179543-rc-... 585268d +/-
airbyte/cloud/models.py — 97% —
airbyte/mcp/_sso_auth.py — 96% —
airbyte/_direct...ctors/models.py — 95% —
airbyte/cloud/client.py — 90% —
airbyte/exceptions.py — 90% —
airbyte/mcp/cloud.py — 85% —
airbyte/progress.py — 80% —
airbyte/cloud/connectors.py — 79% —
airbyte/shared/...ql_processor.py — 68% —
airbyte/_util/api_util.py — 67% —

ℹ️ Calculations represent line coverage. Results may change as calculations are updated.


Updated October 20, 2026 06:18 UTC

@aaronsteers
Aaron ("AJ") Steers (aaronsteers) added this pull request to stack #1398 October 16, 2026 20:01
Comment thread airbyte/cloud/connections.py Outdated
@devin-ai-integration
devin-ai-integration Bot removed this pull request from stack #1398 October 16, 2026 20:38
@devin-ai-integration
devin-ai-integration Bot changed the base branch from devin/1792117913-rc-stream-sync-mode-refresh-catalog to release-candidate/v1 October 16, 2026 21:21
@devin-ai-integration devin-ai-integration Bot changed the title feat(mcp): add stream sync mode overrides to Cloud connections feat(mcp): add batch stream config overrides and update_cloud_connection_unsafe Oct 16, 2026
Comment thread airbyte/cloud/connections.py Outdated
Comment thread airbyte/cloud/models.py Outdated
Comment thread airbyte/mcp/cloud.py Outdated
@aaronsteers
Aaron ("AJ") Steers (aaronsteers) marked this pull request as ready for review October 16, 2026 22:09
Copilot AI balanced review requested due to automatic review settings October 16, 2026 22:09

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 4 potential issues.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

Comment thread airbyte/mcp/cloud.py Outdated
Comment thread airbyte/mcp/cloud.py Outdated
Comment thread airbyte/cloud/connections.py Outdated
Comment thread airbyte/cloud/connections.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Safety decisions can become stale before catalog application, allowing unsafe writes through the safe tool.

3 open issues
What changed in this PR

Adds batch stream configuration updates to Cloud connections, with separate safe and unsafe MCP tools.

Changes:

  • Adds override models, batch validation, and saved-state safety checks.
  • Introduces unsafe updates with reset controls and reporting.
  • Adds tests, tracing metadata, and catalog-cache invalidation.
File Description
tests/​unit_tests/​test_mcp_tool_errors.py Tests unsafe-change error handling.
tests/​unit_tests/​test_mcp_connection_schedules.py Tests tool annotations, resets, and guidance.
tests/​unit_tests/​test_cloud_connection_catalog_edits.py Tests batch validation and catalog updates.
tests/​unit_tests/​fixtures/​arg_trace_plan.json Adds override and unsafe-tool tracing expectations.
airbyte/​mcp/​cloud.py Exposes safe and unsafe batch updates.
airbyte/​mcp/​_error_handling.py Registers unsafe-change errors.
airbyte/​exceptions.py Defines the unsafe-change exception.
airbyte/​cloud/​models.py Defines override and result models.
airbyte/​cloud/​connections.py Implements planning, safety checks, and application.
airbyte/​cloud/​__init__.py Exports stream configuration models.
airbyte/​_util/​api_util.py Adds catalog reset control.

Review effort: Balanced

Comment thread airbyte/cloud/connections.py Outdated
Comment thread airbyte/cloud/connections.py Outdated
Comment thread airbyte/cloud/connections.py Outdated
Comment thread airbyte/cloud/connections.py Outdated
Co-Authored-By: AJ Steers <[email protected]>
Comment thread airbyte/mcp/cloud.py Outdated
Comment thread airbyte/mcp/cloud.py Outdated
devin-ai-integration Bot and others added 2 commits October 16, 2026 22:32
…/1792179543-rc-stream-sync-mode

Co-Authored-By: AJ Steers <[email protected]>

# Conflicts:
#	airbyte/cloud/connections.py
Comment thread airbyte/cloud/connections.py Outdated
Comment thread airbyte/mcp/cloud.py Outdated
Comment thread airbyte/mcp/cloud.py Outdated
Comment thread airbyte/mcp/cloud.py
Comment on lines +4300 to +4303
@mcp_tool(
destructive=True,
)
def update_cloud_connection_unsafe(

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also hide this "unsafe" version of the tool behind the insiders flag for now. And can you confirm our prompts won't direct callers to the unsafe version if it isn't registered based on specific settings?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed on hiding it: update_cloud_connection_unsafe will get insiders_only=True whichever option we pick below.

To confirm your question: today, no. Two places point callers at the unsafe tool without checking whether it's registered:

  1. The override_stream_configs field description on update_cloud_connection says unsafe changes "must be run with update_cloud_connection_unsafe".
  2. The unsafe-change error guidance in _apply_stream_config_overrides says "Retry with update_cloud_connection_unsafe" whenever destructive_tools_allowed(ctx) isn't False. It doesn't check insiders mode, read-only mode, pipeline-change mode, or module/tool exclusions.

Order of operations: the field description is part of the tool schema. It's built once at registration, so it can't vary per request. The error guidance is built at call time, inside the request, so it can check the caller's settings.

Options:

  1. Shared enabledness helper. unsafe_update_tool_enabled(ctx) combines the checks that hide the tool: _insiders_mode, destructive_tools_allowed, pipeline_changes_allowed, and read-only mode. The guidance uses it. Downside: it duplicates the filter chain, so it misses module include/exclude and the standard tool-exclusion list, and it can drift when filters change.
  2. Ask the server at error time. await ctx.fastmcp.list_tools() runs every tool-filter middleware for the current request, so it returns exactly what this caller can see, with nothing duplicated. FastMCP 4.0.9 runs our sync tools in anyio's threadpool, so the sync tool can call it with anyio.from_thread.run. On any failure, the guidance falls back to "use the Airbyte UI". I've confirmed those two FastMCP facts in the installed source, but I haven't proven yet that it works inside a real tool call. A Client test with insiders on and off would prove it.
  3. Tool-neutral description. The description drops the tool name and says something like "Changes that would invalidate a stream's saved state are rejected; the error says how to proceed." The schema then never mentions a tool that may not exist, and the guidance can be exact.
  4. Conditional phrasing everywhere, for example "If update_cloud_connection_unsafe is available, retry with it; otherwise ask the user to make the change in the Airbyte UI." Simplest, but agents may still try the hidden tool.

My pick is 2 + 3, or 1 + 3 if you'd rather not call list_tools from inside a tool. I won't change any code until you pick.

Written by Devin

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is in the stacked PR #1418 at 939d8cb:

  1. update_cloud_connection_unsafe now has insiders_only=True.
  2. The new private _is_unsafe_connection_update_tool_enabled(ctx) checks insiders mode, destructive tools, pipeline changes and read-only mode. The unsafe-change error names the unsafe tool only when the helper returns True. Otherwise it points to the Airbyte UI and gives the connection URL.
  3. The override_stream_configs description no longer names the unsafe tool. It now says the change is rejected and the error explains how to proceed.

To confirm your question: after this change, update_cloud_connection_unsafe appears in the airbyte/ code only in its own definition, the helper's docstring, and the gated guidance string. One gap remains: if a per-tool exclusion list names only this tool, the helper doesn't see it.

Written by Devin

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants