[0.0.5] fix: Python venv, package list and dependency tree for setup.py repos (TC-7405) by tmihalac · Pull Request #492 · RHEcosystemAppEng/exploit-iq-agent

tmihalac · 2026-11-05T09:59:53Z

Problem

TC-7405: a scan of yaml/pyyaml at 5.3 (965d8070, CVE-2020-14343) ends as UNKNOWN/uncertain instead of vulnerable. Every Python dependency step fails, and run_command hides the failures:

INFO  - Detected Python 2.7 from .../setup.py
INFO  - Creating transitive_env with Python 2.7 using uv
ERROR - Failed to run command: ['.../transitive_env/bin/pip', 'list', '--format=freeze']
ERROR - deptree returned empty output
FileNotFoundError: [Errno 2] No such file or directory: '.../requirements.txt'

The ticket also says uv is missing from the image. That is not the case for quay.io/ecosystem-appeng/exploit-iq-agent:0.0.5: uv 0.7.15 is at /usr/bin/uv, and shutil.which("uv") finds it.

Causes and changes

All reproduced in exploit-iq-agent:0.0.5, run from /workspace with an arbitrary UID.

Python 2.7 detected. pyyaml declares python_requires='>=2.7, !=3.0.*, ...', so determine_python_version() returns 2.7. uv venv --python 2.7 fails with Python <3.7 is not supported but 2.7 was requested and no venv is created. Change: _ensure_venv retries with the current interpreter when the venv is still missing, and logs an error when both attempts fail.
Exit codes ignored. run_command logged Successfully ran command for every command and put stderr at DEBUG. Change: a non-zero exit logs a WARNING with the exit code; stderr stays at DEBUG, because the E2E log scan treats WARNING lines that contain "error" as failures and npm ls prints error: on expected exits. stdout is still returned, so callers behave as before.
No bin/pip. uv venv does not install pip, so installed_packages.txt was never written. Change: use uv pip list --format=freeze --python <venv python>.
Agent uv constraints applied to the analyzed repo. From /workspace, uv reads the agent's [tool.uv] constraint-dependencies (setuptools>=84.0.0), which conflicts with setuptools<81, so deptree is never installed. Change: backport of fix: install deptree with --no-config so agent uv constraints do not apply #456 (--no-config) and fix: use --no-dev (and no shell redirect) in the lock-file uv export #457 (uv export ... --no-dev, no shell redirect), cherry-picked with -x. The fix: install deptree with --no-config so agent uv constraints do not apply #456 conflict is resolved by keeping python -m deptree, which works on 0.0.5.
requirements.txt crash. build_tree opened it unconditionally. Change: without requirements.txt, the direct dependencies come from Requires-Dist of the repo's own distribution in transitive_env. uv pip install . installs that distribution, and its PEP 610 direct_url.json points at the repo. The lookup searches both lib and lib64, because RHEL installs platform wheels under lib64 (pyyaml does). Lock-file and Pipfile repos get no direct dependencies marked, and they no longer crash.
Wrong import name. The re-key used the first line of top_level.txt, so pyyaml (_yaml, yaml) became _yaml. Change: prefer the module named like the distribution, then the first public module. Also read top_level.txt from transitive_env before the agent's own environment.

Verification

Unit tests in test_python_build_tree.py: 26 pass. Each new test failed before the change.
New test_python_dep_tree_real_uv.py clones pyyaml at 965d8070 and runs the real install_dependencies() and build_tree() with real uv from the agent directory. It fails on the old code (no venv) and passes now. It skips when uv or git is missing or the clone fails.
In the 0.0.5 image, with the changed dep_tree.py mounted: the venv is created with Python 3.12 after the 2.7 attempt fails (logged), installed_packages.txt is pyyaml==5.3, deptree runs, and the tree has yaml, not _yaml.
End-to-end: the Tekton integration test on this PR runs the new pyyaml 5.3 / CVE-2020-14343 case from ci/it/integration-tests-input.json. 0.0.5 has no E2E workflows or scripts/e2e: the GitHub Kustomize E2E of this PR checks out its workflow, harness and fixture from main (run log of [0.0.5] ci: build release-branch images (backport of #474 and #477) #491: +88d7a8ec...:refs/remotes/origin/main). The E2E test for this scenario (a dedicated step that runs the pyyaml case and checks the engine log with check_python_dependency_logs.py) is therefore added on main in fix: Python venv, package list and dependency tree for setup.py repos (TC-7405) #493, and in the operator OLM E2E in test(e2e): pyyaml 5.3 setup.py-only case with engine log check in the OLM E2E (TC-7405) exploit-iq-operator#331. After fix: Python venv, package list and dependency tree for setup.py repos (TC-7405) #493 merges, /test-e2e on this PR runs it against this PR's agent image.
I have not run the full pipeline (with the LLM) myself, so the final vulnerable verdict is not verified here. The integration test on this PR checks it.

🤖 Generated with Claude Code

tmihalac · 2026-11-05T11:48:45Z

/retest

tmihalac · 2026-11-05T12:18:06Z

/retest

tmihalac · 2026-11-05T12:44:22Z

/retest

tmihalac · 2026-11-05T13:21:31Z

/retest

tmihalac · 2026-11-05T13:22:00Z

/retest

tmihalac · 2026-11-05T13:27:23Z

/retest

tmihalac · 2026-11-05T13:34:17Z

/e2e-test

tmihalac · 2026-11-05T14:00:48Z

/retest

tmihalac · 2026-11-05T14:03:20Z

/e2e-test

tmihalac · 2026-11-05T14:15:52Z

/e2e-test

tmihalac · 2026-11-05T14:17:54Z

/test-e2e

tmihalac · 2026-11-05T14:27:13Z

/retest

tmihalac · 2026-11-05T14:28:23Z

/test-e2e

tmihalac · 2026-11-05T14:58:53Z

/retest

tmihalac · 2026-11-05T15:00:02Z

/test-e2e

tmihalac · 2026-11-05T15:26:18Z

/retest

tmihalac · 2026-11-05T16:02:43Z

/test-e2e

tmihalac · 2026-11-05T16:04:52Z

/test-e2e

tmihalac · 2026-11-05T16:23:39Z

/test-e2e

…apply (RHEcosystemAppEng#456) * fix: install deptree with --no-config so agent uv constraints do not apply Python dependency trees fell back to requirements.txt ("Using requirements.txt fallback for dependency tree (2 packages)"): the deptree install uv pip install 'setuptools<81' deptree --python <repo>/transitive_env/bin/python runs from /workspace, so uv applies the agent's own [tool.uv] constraint-dependencies (setuptools>=84.0.0, added in RHEcosystemAppEng#372) to the analyzed repo's venv and fails with "Because you require setuptools<81 and setuptools>=84.0.0, we can conclude that your requirements are unsatisfiable". bin/deptree is never created. run_command ignores the exit code and logs "Successfully ran command", with stderr only at DEBUG, so the failure was silent. Found by the live E2E (Python case, CVE-2024-49767), whose log scan flags ERROR lines. Pass --no-config. Verified: from a directory with the agent pyproject.toml the install fails and bin/deptree is missing; with --no-config it installs. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> * fix: pass --no-config to the other uv pip install calls that run from /workspace Same root cause as the deptree install: uv applies the agent's own [tool.uv] constraint-dependencies (urllib3>=2.8.0, click>=8.3.3, protobuf>=5.29.6, orjson>=3.11.6, setuptools>=84.0.0) to the analyzed repo's venv, so packages the repo pins lower fail with "No solution found". Reproduced with real uv: urllib3==1.26.5, click==7.1.2, protobuf==3.20.1, orjson==3.9.0 and requests==2.25.0 (needs urllib3<1.27) fail; with --no-config they install. Calls changed: install_dependency (requirements.txt, per line), and the "uv pip install -r -" calls for lock-file and Pipfile repos. "uv pip install ." keeps cwd=<repo>, where uv reads no agent config (verified: installs the repo's exact pins). Co-Authored-By: Claude Sonnet 5.5 <[email protected]> --------- Co-authored-by: Claude Sonnet 5.5 <[email protected]> (cherry picked from commit 4da701a)

…HEcosystemAppEng#457) The agent ran uv export --format requirements-txt no-dev 2>/dev/null through run_command (args list, no shell). uv exits 2 with "error: unexpected argument 'no-dev' found" and no output, run_command returns "" (not None), and the code then runs "uv pip install -r -" with empty input and reports the lock file as installed. Repos with a uv.lock therefore installed nothing. Use "uv export --format requirements-txt --no-dev". Verified with real uv: the old command exits 2; the fixed command exports the locked pins. Not changed: for poetry.lock repos "uv export" fails with "No `project` table found" (a classic Poetry project is not a uv project), and the pipenv call has the same literal "2>/dev/null" argument. Co-authored-by: Claude Sonnet 5.5 <[email protected]> (cherry picked from commit 61c79aa)

…p.py repos (TC-7405) A scan of pyyaml 5.3 (CVE-2020-14343) ended as UNKNOWN/uncertain because every Python dependency step failed, and the failures were silent: - determine_python_version() returns "2.7" for python_requires='>=2.7, !=3.0.*, ...'. `uv venv --python 2.7` fails ("Python <3.7 is not supported") and no venv is created. _ensure_venv now retries with the current interpreter when the venv is missing after the first attempt, and logs an error when both attempts fail. - run_command ignored the exit code and logged "Successfully ran command". A non-zero exit now logs a WARNING with the exit code and stderr. stdout is still returned, so callers keep their behavior. - installed_packages.txt was built with transitive_env/bin/pip, which `uv venv` never creates. It now uses `uv pip list --format=freeze`. - build_tree opened requirements.txt unconditionally and raised FileNotFoundError for repos with only setup.py/setup.cfg/pyproject.toml. Without requirements.txt, the direct dependencies now come from the Requires-Dist of the repo's own distribution in transitive_env (found by its PEP 610 direct_url.json, under lib or lib64). - The re-key to import names took the first line of top_level.txt, so pyyaml ("_yaml", "yaml") became "_yaml". It now prefers the module named like the distribution, then the first public module, and reads top_level.txt from transitive_env before the agent's own environment. Verified in quay.io/ecosystem-appeng/exploit-iq-agent:0.0.5 with the changed dep_tree.py mounted, from /workspace: the venv is created with Python 3.12, installed_packages.txt holds pyyaml==5.3, deptree runs and the tree has "yaml". Co-Authored-By: Claude Opus 5.5 <[email protected]>

…est (TC-7405) pyyaml 5.3 declares a broad python_requires and has no requirements.txt. The case is expected to be vulnerable (unsafe full_load/FullLoader). Co-Authored-By: Claude Opus 5.5 <[email protected]>

The E2E log scan treats a WARNING line that contains "error" as a failure. Tools print "error:" to stderr even for expected non-zero exits (npm ls), so the WARNING now names only the command and exit code; stderr stays at DEBUG. Co-Authored-By: Claude Opus 5.5 <[email protected]>

…de the repo (TC-7405) The integration test on PR 492 failed the pyyaml 5.3 case (label false_positive, "pyyaml not found in dependencies"). The engine passes the repo as a path relative to /workspace (.cache/am_cache/git/...), and _ensure_venv returned a relative venv python. `uv pip install . --python <venv python>` runs with cwd=<repo>, so uv looked the path up inside the repo and failed with "No virtual environment or system Python installation found for path"; pyyaml was never installed and installed_packages.txt stayed empty. _ensure_venv now returns an absolute path. The real-uv test now passes the repo as a relative path from the agent directory, as the engine does; it failed before this change. Co-Authored-By: Claude Opus 5.5 <[email protected]>

…own directory has a version (TC-7405) The pyyaml 5.3 case (CVE-2020-14343) still failed the integration test on PR 492 with false_positive: "Version check completed - 1 package(s) matched but none are vulnerable ... stdlib:None (unknown)". pyyaml has only setup.py, so `uv pip install .` installs pyyaml itself into transitive_env, and `uv pip freeze` prints it as "pyyaml @ file:///..." without a version. parse_python_venv skipped such lines, so the version check found no pyyaml and fell back to the stdlib check. It now runs `uv pip list --format=freeze`, which prints "pyyaml==5.3". Verified with real uv: freeze prints "pyyaml @ file:///tmp/...", list prints "pyyaml==5.3". The real-uv pyyaml test now also checks parse_python_venv. Co-Authored-By: Claude Opus 5.5 <[email protected]>

…o is the vulnerable Python package (TC-7405) The pyyaml 5.3 scan (CVE-2020-14343) ended as code_not_reachable once the dependency steps worked: every Call Chain Analyzer query was negative. - chain_of_calls_retriever: "PyYAML,yaml.full_load" read the module name "yaml" as a class name and logged "Cannot find initial function=full_load". For Python only, when the class lookup finds nothing and the qualifier is the package or one of its modules, look the function up without a class. - dep_tree: the repo is pyyaml itself, and its own distribution had no parent ("yaml" -> []), so repo code that calls it (examples/yaml-highlight/yaml_hl.py) was never searched. List the repo's own distribution with its Requires-Dist, so it becomes a child of root_project. - python_functions_parser: Python 3 cannot parse the Python 2 sources in pyyaml's lib/; the SyntaxError failed the whole query. Such a caller is not a caller. Co-Authored-By: Claude Opus 5.5 <[email protected]>

tmihalac · 2026-11-05T19:15:20Z

/retest

exploit-iq-pac · 2026-11-05T19:15:29Z

All PipelineRuns for this commit have already succeeded. Use /retest <pipeline-name> to re-run a specific pipeline or /test to re-run all pipelines.

tmihalac · 2026-11-05T19:15:57Z

/test

tmihalac · 2026-11-05T19:16:26Z

/test-e2e

tmihalac · 2026-11-05T19:20:47Z

/test

tmihalac · 2026-11-05T19:28:33Z

/test vulnerability-analysis-on-pr

tmihalac · 2026-11-05T19:32:10Z

/test-e2e

tmihalac · 2026-11-05T19:59:26Z

/test

tmihalac · 2026-11-05T20:27:59Z

/test-e2e

tmihalac · 2026-11-05T21:00:24Z

/test-e2e

tmihalac mentioned this pull request Nov 5, 2026

fix: Python venv, package list and dependency tree for setup.py repos (TC-7405) #493

Merged

tmihalac closed this Nov 5, 2026

tmihalac reopened this Nov 5, 2026

tmihalac mentioned this pull request Nov 5, 2026

fix: absolute venv python so uv pip install . finds it inside the repo (TC-7405) #494

Merged

tmihalac mentioned this pull request Nov 5, 2026

[0.0.5] feat: create NetworkPolicies for the ExploitIQ operands (TC-6045) RHEcosystemAppEng/exploit-iq-operator#334

Merged

tmihalac mentioned this pull request Nov 5, 2026

[0.0.5] feat: create NetworkPolicies for the ExploitIQ operands (TC-6045) RHEcosystemAppEng/exploit-iq-operator#329

Closed

tmihalac mentioned this pull request Nov 5, 2026

fix: read the venv with uv pip list so the repo installed from its own directory has a version (TC-7405) #495

Merged

tmihalac mentioned this pull request Nov 5, 2026

fix: Call Chain Analyzer reaches the vulnerable function when the repo is the vulnerable Python package (TC-7405) #496

Merged

tmihalac closed this Nov 5, 2026

tmihalac reopened this Nov 5, 2026

tmihalac and others added 7 commits November 5, 2026 20:20

test(it): add the pyyaml 5.3 CVE-2020-14343 case to the integration t…

aae614c

…est (TC-7405) pyyaml 5.3 declares a broad python_requires and has no requirements.txt. The case is expected to be vulnerable (unsafe full_load/FullLoader). Co-Authored-By: Claude Opus 5.5 <[email protected]>

tmihalac force-pushed the fix-TC-7405-0.0.5 branch from 69d015d to 208da33 Compare November 5, 2026 18:20

tmihalac closed this Nov 5, 2026

tmihalac reopened this Nov 5, 2026

tmihalac merged commit 5de0a58 into RHEcosystemAppEng:0.0.5 Nov 6, 2026
17 of 29 checks passed

tmihalac deleted the fix-TC-7405-0.0.5 branch November 6, 2026 05:20

Conversation

tmihalac commented Nov 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Causes and changes

Verification

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

exploit-iq-pac Bot commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

tmihalac commented Nov 5, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

tmihalac commented Nov 5, 2026 •

edited

Loading