fix(cloud): tolerate undecodable mappers and unknown statuses in connection/job lookups by aaronsteers · Pull Request #1335 · airbytehq/PyAirbyte

Aaron ("AJ") Steers (aaronsteers) · 2026-10-06T01:18:26Z

Summary

👉 TL;DR: Cloud connection and job lookups no longer fail outright when the bundled `airbyte-api` SDK (0.53.0) can't decode part of the response. This covers 1Password's Marketo/PartnerStack connections (`field-filtering` mappers), `locked` connections, and job statuses the SDK doesn't define. When the SDK raises `ValueError` while decoding, we log a warning, re-fetch the raw Public API JSON, and decode it leniently: we drop `configurations.streams[*].mappers`, and we keep any status value the SDK enum doesn't define as a plain string. This PR adds no SDK monkeypatching. The `queued` job status is now registered on the SDK enum by #1334 (merged), so `queued` decodes on the first request.

Changes

get_connection / list_connections: on ValueError, fall back to GET /connections/{id} (or the per-page GET /connections) and _lenient_decode_connection.
get_job_info / get_job_logs: on ValueError, fall back to GET /jobs/{id} (or the per-page GET /jobs) and _lenient_decode_job. Pagination is unchanged and reads from either source.
New private helpers:
- _get_public_api_json: a bounded-timeout requests GET using _config_api_headers. HTTP 404 raises AirbyteMissingResourceError and 403 keeps the existing forbidden guidance, both matching _wrap_sdk_error.
- _decode_with_unknown_status: decodes with a known placeholder status, then calls dataclasses.replace(status=raw_status).
- _lenient_decode_connection and _lenient_decode_job.
Public enums: ConnectionStatus.LOCKED added. JobStatusEnum.QUEUED came from fix(cloud): support the queued job status #1334. Both are additive, and both values are in the platform spec.
Unit tests (responses-mocked so the real SDK decode raises):
- field-filtering mapper (get + list)
- a job status unknown to the SDK (get + list)
- locked connection
- fallback 404 → AirbyteMissingResourceError
- the no-fallback path

Review Spotlight

_decode_with_unknown_status / _lenient_decode_connection / _lenient_decode_job at the bottom of airbyte/_util/api_util.py.
get_job_logs restructure: SDK and fallback paths both feed page_data / next_page.

Risks

Only in the fallback path: mappers are dropped (nothing in PyAirbyte reads them), and status may hold a raw str instead of an SDK enum. Public CloudConnectionInfo / CloudJobInfo deliberately keep closed status enums, so a status unknown to our public enums still raises there; we'll add new values as the platform introduces them.
The fallback catches any ValueError from the SDK call. If the lenient re-decode also fails, that error propagates as before.
No dependency or lock changes.

Follow-ups

Durable fix: replace Speakeasy with airbyte-server-models ( refactor(cloud): replace Speakeasy airbyte-api SDK with generated airbyte-server-models #1148 ). The airbyte-api 1.0.1 bump ( chore: adapt PyAirbyte to airbyte-api==1.0.1 breaking changes #1048 ) still fails with Could not find discriminator field algorithm.

Test plan

uv run ruff check . && uv run ruff format --check .
uv run pyrefly check: 0 errors
uv run pytest tests/unit_tests/test_cloud_api_util.py -q: 108 passed (after merging release-candidate/v1)

Link to Devin session: https://app.devin.ai/sessions/b71c6ee648ab4a67b4e68990fc713daf
Open in Devin Desktop: https://app.devin.ai/desktop/session/b71c6ee648ab4a67b4e68990fc713daf?variant=devin
Requested by: Aaron ("AJ") Steers (@aaronsteers)

Co-Authored-By: AJ Steers <[email protected]>

devin-ai-integration · 2026-10-06T01:18:29Z

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

Disable automatic comment, CI, and merge conflict monitoring

github-actions · 2026-10-06T01:18:50Z

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This PyAirbyte Version

You can test this version of PyAirbyte using the following:

# Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1791249227-graceful-connection-decode' pyairbyte --help

# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1791249227-graceful-connection-decode'

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

/fix-pr - Fixes most formatting and linting issues
/uv-lock - Updates uv.lock file
/test-pr - Runs tests with the updated PyAirbyte
/prerelease - Builds and publishes a prerelease version to PyPI

📚 Show Repo Guidance

Helpful Resources

Community Support

Questions? Join the #pyairbyte channel in our Slack workspace.

📝 Edit this welcome message.

Co-Authored-By: AJ Steers <[email protected]>

github-code-quality · 2026-10-06T01:28:22Z

Code Coverage Overview

Languages: Python

Python / code-coverage/pytest-no-creds

The overall line coverage in commit 05673e8 in the devin/1791249227-gra... branch is 77%. The line coverage in commit d9f652f in the release-candidate/v1 branch is 65%.

Show a line coverage summary of the most impacted files.

File	`release-candidate/v1` d9f652f	`devin/1791249227-gra...` 05673e8	+/-
`airbyte/_util/api_util.py`	36%	65%	+29%
`airbyte/mcp/cloud.py`	52%	82%	+30%
`airbyte/cloud/connectors.py`	34%	75%	+41%
`airbyte/mcp/int..._registry_ui.py`	0%	92%	+92%
`airbyte/mcp/_user_identity.py`	0%	93%	+93%
`airbyte/_direct...ctors/models.py`	0%	94%	+94%
`airbyte/mcp/_telemetry.py`	0%	94%	+94%
`airbyte/mcp/_sso_auth.py`	0%	96%	+96%
`airbyte/cloud/models.py`	0%	97%	+97%
`airbyte/mcp/int...nc_status_ui.py`	0%	97%	+97%

Python / code-coverage/pytest

The overall line coverage in commit 05673e8 in the devin/1791249227-gra... branch is 80%. The line coverage in commit d9f652f in the release-candidate/v1 branch is 71%.

Show a line coverage summary of the most impacted files.

File	`release-candidate/v1` d9f652f	`devin/1791249227-gra...` 05673e8	+/-
`airbyte/cloud/connectors.py`	51%	80%	+29%
`airbyte/mcp/cloud.py`	52%	82%	+30%
`airbyte/mcp/_otel.py`	0%	78%	+78%
`airbyte/mcp/int..._registry_ui.py`	0%	92%	+92%
`airbyte/mcp/_user_identity.py`	0%	93%	+93%
`airbyte/_direct...ctors/models.py`	0%	94%	+94%
`airbyte/mcp/_telemetry.py`	0%	94%	+94%
`airbyte/mcp/_sso_auth.py`	0%	96%	+96%
`airbyte/cloud/models.py`	0%	97%	+97%
`airbyte/mcp/int...nc_status_ui.py`	0%	97%	+97%

Python / code-coverage/pytest-unit

The overall line coverage in commit 05673e8 in the devin/1791249227-gra... branch is 74%. Line coverage data for the release-candidate/v1 branch is not yet available.

Show a line coverage summary of the most covered files.

File	`release-candidate/v1`	`devin/1791249227-gra...` 05673e8	+/-
`airbyte/cloud/models.py`	—	97%	—
`airbyte/mcp/_sso_auth.py`	—	96%	—
`airbyte/_direct...ctors/models.py`	—	94%	—
`airbyte/cloud/client.py`	—	90%	—
`airbyte/exceptions.py`	—	85%	—
`airbyte/mcp/cloud.py`	—	82%	—
`airbyte/progress.py`	—	80%	—
`airbyte/cloud/connectors.py`	—	75%	—
`airbyte/shared/...ql_processor.py`	—	68%	—
`airbyte/_util/api_util.py`	—	65%	—

_{Updated October 07, 2026 20:32 UTC}

…cked) Co-Authored-By: AJ Steers <[email protected]>

devin-ai-integration

Devin Review found 2 potential issues.

Co-Authored-By: AJ Steers <[email protected]>

Aaron ("AJ") Steers (aaronsteers) · 2026-10-07T20:18:26Z

Devin, update this PR from the base branch. Note the related PR has mergec.

…/1791249227-graceful-connection-decode Co-Authored-By: AJ Steers <[email protected]> # Conflicts: # airbyte/mcp/cloud.py # tests/unit_tests/test_cloud_api_util.py

Copilot

🟢 Approval recommended

The fallback behavior is bounded, preserves existing paths, and has focused regression coverage.

0 open findings

🧠 Review effort: Balanced

fix(cloud): tolerate undecodable stream mappers in connection lookups

b1024f5

Co-Authored-By: AJ Steers <[email protected]>

devin-ai-integration Bot assigned Aaron ("AJ") Steers (aaronsteers) Oct 6, 2026

Airbyte Support Bot (airbyte-support-bot) requested a balanced review from Copilot October 6, 2026 01:18

Copilot started reviewing on behalf of Airbyte Support Bot (airbyte-support-bot) October 6, 2026 01:19

View session

This comment was marked as resolved.

Sign in to view

fix(cloud): bound timeouts on public API fallback GET

1bc0dba

Co-Authored-By: AJ Steers <[email protected]>

fix(cloud): tolerate unknown job/connection status values (queued, lo…

27efe21

…cked) Co-Authored-By: AJ Steers <[email protected]>

devin-ai-integration Bot changed the title ~~fix(cloud): tolerate undecodable stream mappers in get_connection / list_connections~~ fix(cloud): tolerate undecodable mappers and unknown statuses in connection/job lookups Oct 6, 2026

Airbyte Support Bot (airbyte-support-bot) requested a balanced review from Copilot October 6, 2026 01:59

Copilot started reviewing on behalf of Airbyte Support Bot (airbyte-support-bot) October 6, 2026 02:00

View session

This comment was marked as resolved.

Sign in to view

devin-ai-integration Bot marked this pull request as ready for review October 6, 2026 02:23

devin-ai-integration Bot reviewed Oct 6, 2026

View reviewed changes

Comment thread airbyte/_util/api_util.py

Comment thread airbyte/_util/api_util.py

fix(cloud): map fallback 404 to AirbyteMissingResourceError

551d119

Co-Authored-By: AJ Steers <[email protected]>

Merge remote-tracking branch 'origin/release-candidate/v1' into devin…

05673e8

…/1791249227-graceful-connection-decode Co-Authored-By: AJ Steers <[email protected]> # Conflicts: # airbyte/mcp/cloud.py # tests/unit_tests/test_cloud_api_util.py

Aaron ("AJ") Steers (aaronsteers) requested a balanced review from Copilot October 7, 2026 20:25

Copilot started reviewing on behalf of Aaron ("AJ") Steers (aaronsteers) October 7, 2026 20:26

View session

Copilot AI reviewed Oct 7, 2026

View reviewed changes

Aaron ("AJ") Steers (aaronsteers) merged commit 9bd5637 into release-candidate/v1 Oct 7, 2026
28 checks passed

Aaron ("AJ") Steers (aaronsteers) deleted the devin/1791249227-graceful-connection-decode branch October 7, 2026 21:09

Navigation Menu

Repository navigation

fix(cloud): tolerate undecodable mappers and unknown statuses in connection/job lookups - #1335

fix(cloud): tolerate undecodable mappers and unknown statuses in connection/job lookups#1335
Aaron ("AJ") Steers (aaronsteers) merged 5 commits into
release-candidate/v1from
devin/1791249227-graceful-connection-decode

Aaron ("AJ") Steers (aaronsteers) commented Oct 6, 2026 •

edited by devin-ai-integration Bot

Loading

Uh oh!

devin-ai-integration Bot commented Oct 6, 2026

Uh oh!

github-actions Bot commented Oct 6, 2026

Testing This PyAirbyte Version

PR Slash Commands

Helpful Resources

Community Support

Uh oh!

This comment was marked as resolved.

Uh oh!

github-code-quality Bot commented Oct 6, 2026 •

edited

Loading

Uh oh!

This comment was marked as resolved.

Uh oh!

devin-ai-integration Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Aaron ("AJ") Steers (aaronsteers) commented Oct 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Aaron ("AJ") Steers (aaronsteers) commented Oct 6, 2026 • edited by devin-ai-integration Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Review Spotlight

Risks

Follow-ups

Test plan

Uh oh!

devin-ai-integration Bot commented Oct 6, 2026

Uh oh!

github-actions Bot commented Oct 6, 2026

👋 Greetings, Airbyte Team Member!

Testing This PyAirbyte Version

PR Slash Commands

Helpful Resources

Community Support

Uh oh!

This comment was marked as resolved.

Uh oh!

github-code-quality Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Overview

Python / code-coverage/pytest-no-creds

Python / code-coverage/pytest

Python / code-coverage/pytest-unit

Uh oh!

This comment was marked as resolved.

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Aaron ("AJ") Steers (aaronsteers) commented Oct 7, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Aaron ("AJ") Steers (aaronsteers) commented Oct 6, 2026 •

edited by devin-ai-integration Bot

Loading

github-code-quality Bot commented Oct 6, 2026 •

edited

Loading