Skip to content

Tell agents to stop when breeze cannot run, not use a host venv - #75599

Merged
potiuk merged 2 commits into
apache:mainfrom
shahar1:docs/agents-breeze-unavailable
Oct 27, 2026
Merged

potiuk merged 2 commits into
apache:mainfrom
shahar1:docs/agents-breeze-unavailable

Conversation

@shahar1

@shahar1 shahar1 commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

This PR resolves a contradiction in AGENTS.md: it forbids host pytest but also documents specific uv run --project ... pytest checks, so the PR names those checks as the only exception.
For Breeze-dependent provider tests, agents must stop and report a gateway refusal instead of building a substitute host environment.
Adds the documented Breeze route for testing against a released Airflow version and clarifies that Magpie’s Breeze permission does not bypass gateway restrictions.

AI Summary

A sandboxed agent was asked to test the Google provider against Airflow 2.11.1. Its breeze calls were refused by the magpie container gateway, which by design does not proxy BuildKit sessions or unlabelled docker inspect lookups. The agent then quietly built a host virtualenv with Airflow 2.11.1 instead of reporting the failure. Two things in AGENTS.md allowed this. It said "never run pytest on the host" while also listing uv run --project ... pytest commands. It also gave no guidance for when breeze is unavailable, and no command for running provider tests against a released Airflow version.

This PR:

  • names the listed uv run --project commands as the only exception to the host rule;
  • tells agents that when a step needs breeze and breeze cannot run, they stop and report which call failed, rather than building a substitute host environment;
  • adds the documented breeze shell --use-airflow-version <VERSION> --mount-sources providers-and-tests route to the generated AGENTS.md commands, via an AGENT-SKILL block in contributing-docs/testing/unit_tests.rst;
  • notes in .apache-magpie-overrides/setup-isolated-setup-install.md that the breeze permission allows only skip the prompt and do not make breeze work behind the container gateway.

Checks run locally:

  • prek run --from-ref upstream/main --stage pre-commit passed, including "Sync AGENTS.md commands from contributing docs" and "Check documented breeze and prek commands exist".
  • The manual stage passed.
  • Skipped: lychee-docker, because Docker was unavailable in the session.
  • Skipped: check-eval-hash. The skill-eval has not been re-run yet, so dev/skill-evals/last-eval-hash.txt is stale and this PR stays a draft until that hash is updated.

The companion magpie change that hints agents on gateway policy refusals is apache/magpie#1810.


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5.5, Sonnet 5.5)

Generated-by: Claude Code (Opus 5.5, Sonnet 5.5) following the guidelines

@shahar1 shahar1 changed the title Tell agents to stop when breeze cannot run instead of using a host venv Tell agents to stop when breeze cannot run, not use a host venv Oct 25, 2026
@shahar1
shahar1 marked this pull request as ready for review October 25, 2026 06:56
@shahar1
shahar1 marked this pull request as draft October 25, 2026 07:33

@amoghrajesh amoghrajesh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intentionally draft?

AGENTS.md said never to run pytest on the host but also listed
`uv run --project` commands, and had no guidance for when breeze is
unavailable. A sandboxed agent whose breeze calls were refused by the
container gateway built a host virtualenv with Airflow 2.11.1 instead of
reporting the failure.

Narrow the host exception to the listed `uv run` commands, tell agents
to stop and report when breeze cannot run, and document the breeze
command for testing providers against a released Airflow version
(generated from contributing-docs). Note in the magpie override that the
breeze allows do not make breeze work behind the container gateway.
@shahar1
shahar1 force-pushed the docs/agents-breeze-unavailable branch from d829d4c to 0620325 Compare October 27, 2026 07:28
@shahar1
shahar1 marked this pull request as ready for review October 27, 2026 07:30
@shahar1

shahar1 commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Intentionally draft?

Had CI failures, should be good now

@shahar1 shahar1 added the ready for maintainer review Set by triage when all criteria pass, or by the fork CI reconciler when fork CI is green. label Oct 27, 2026
@potiuk
potiuk merged commit be28f7f into apache:main Oct 27, 2026
85 of 97 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:dev-tools ready for maintainer review Set by triage when all criteria pass, or by the fork CI reconciler when fork CI is green.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants