Skip to content

test: prune audited duplicate e2e tests / test: 删除已审计的重复端到端测试 - #3574

Merged
functionstackx merged 1 commit into
mainfrom
test/prune-audited-e2e-tests
Sep 29, 2026
Merged

functionstackx merged 1 commit into
mainfrom
test/prune-audited-e2e-tests

Conversation

@functionstackx

@functionstackx functionstackx commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Delete 36 redundant or self-comparing test declarations across 12 E2E test files, plus their unused test helper/imports. This is the reviewed deletion-only subset of the exhaustive test audit, not a blanket removal of every candidate.

479 test/support lines deleted; zero production, configuration, or workflow changes. All 1,226 surviving audited test declarations are unchanged, including decorators.

Companion app cleanup: InferenceX-app #1231. The changes are independently mergeable and neither changes Pareto behavior.

Test area Declarations removed Existing proof retained
Eval shell dispatch 6 Real Kimi/BFCL runner paths, environment precedence, provider metadata, and unsupported-suite rejection
Golden acceptance lengths 1 Controlled synthetic-acceptance values; committed-curve validation remains
Matrix generation 2 Exact generated name/order assertions and real CLI conflicting-flag rejection
AgentX power window 1 Persisted multinode window with the same boundaries/counts/token totals
AgentX server-log metrics 4 Real processor output covering duplicate engine tags, SGLang DP capacity and multiple worker logs
Power column parsing/accumulator 4 Real telemetry integration and published accumulator-check output
Eval collection 3 Batched suite identity, compatibility-format ingestion and exact timestamp fallback
Reusable-run selection 2 Real selector expired-artifact handling and acknowledgment ancestry rejection
Merge orchestration 9 Failed-old/success-new reruns, real transient retry, SHA-pinned squash, CLI errors and branch lifecycle
Failed-ingest recovery 2 Full inspection/metadata output and shared duplicate-YAML rejection
Sign-off publication 1 Trusted resource-specific comment publication with spoofed-marker protection
Reuse eval identity 1 Multi-task artifact validation through the actual duplicate-identity checks

Remove _run_full_flow after its sole test caller disappears, the empty TestExitCodes container, and four unused imports. No production API or export becomes dead in this batch.

Explicitly retained

Preservation review reduced the original 51 deletion candidates to 36. Keep:

  • Ten tests for the legacy aggregate_power compatibility API until API retirement is separately decided.
  • Two matrix malformed-input characterization tests.
  • The token-secrecy test until its assertions are transferred to an equivalent owner.
  • Initial merge-state safety: the proposed lifecycle keeper overwrites the initializer before checking cleanup.
  • Validation of every committed golden-data cell: controlled temporary curves do not replace shipped-resource validation.

The 146 consolidation candidates and 73 assertion repairs from the audit are not applied here. The separately reproduced SWE-bench predictions-file decoder bug is also out of scope.

Validation

  • Pinned baseline dc911f6d8e488848f06809b430ecb5efd6c2477c: 2,151 passed, one skipped, 199 subtests passed.

  • After deletions: 2,114 passed, one skipped, 199 subtests passed. The 36 declarations represent 37 collected cases; the skip still requires CUDA/ROCm.

    uv run --project inferencex-e2e --locked --all-extras --group test --no-editable \
      python -m pytest -c inferencex-e2e/pyproject.toml \
      inferencex-e2e/infx/tests/ inferencex-e2e/utils/ inferencex-e2e/runners/ \
      collectivex/tests/ operatorx/tests/ \
      --ignore=inferencex-e2e/utils/srt-slurm -n 4 -q
  • Repository Ruff lint/format checks and git diff --check passed.

  • Independent pre-deletion and final preservation reviews: no P0–P2 findings in the final diff. AST/source reconciliation confirmed exactly the approved removals, all 15 exclusions retained, and no changes to surviving declarations.

  • The OpenClaw Codex autoreview helper was attempted on this diff but returned reviewer_unavailable / engine_failed without a report. The independent review is a disclosed fallback, not a successful Codex run.

  • No GPU sweep or eval dispatch; no performance changelog entry is needed for test-only deletions.

AI model disclosure

  • GPT 6 Astra Fast: implementation, validation, reconciliation and PR preparation; identifier supplied by the runtime.
  • Eight owner-lane audit agents: exhaustive read-only discovery and test ledgers. Their exact model/version was not exposed and could not be verified.
  • One independent preservation reviewer: pre-deletion coverage challenge and final diff review. Its exact model/version was not exposed and could not be verified.

Type of Change

  • Other: deletion-only test maintenance.

Checklist

  • AI model disclosure completed.
  • Full local CPU suite run.
  • Surviving coverage independently reviewed.
  • Documentation, performance changelog and sweep reuse: not applicable.
  • PR creation only; no merge requested.
中文

改动说明

从全面测试审计中选择已核实的删除项:删除 12 个 E2E 测试文件中的 36 个重复或自比较测试声明,以及失去调用方的测试辅助方法、空测试类和未使用导入。测试及辅助代码共减少 479 行;生产代码、配置和工作流均无改动。其余 1,226 个已审计测试声明及装饰器保持不变。

保留的测试通过实际处理器、CLI、工作流调用或持久化产物验证相关行为。上方表格列出各模块的删除数量和保留依据。

覆盖保全审查把 51 个候选项缩减为 36 个:暂时保留旧 aggregate_power API 的十个测试、两个矩阵异常输入测试、一个令牌保密测试、初始合并状态安全测试,以及全部已提交 golden 数据的校验测试。其替代覆盖或兼容性决策尚不充分。

本 PR 不处理审计中的 146 个合并候选项、73 个断言修复项,也不混入已复现的 SWE-bench 预测文件解码缺陷。

对应的 app 清理见 InferenceX-app #1231。两个 PR 可独立合并,均不改变 Pareto 行为。

验证与审查

基线:2,151 个测试和 199 个子测试通过,跳过一个需要 CUDA/ROCm 的测试。修改后:2,114 个测试和 199 个子测试通过,仍跳过同一个 GPU 测试。36 个声明对应 37 个参数化测试实例。

仓库 Ruff 检查、格式检查和 git diff --check 通过。独立删除前审查及最终覆盖保全审查未发现最终差异中的 P0–P2 问题;结构与源码比对确认只删除核准项,15 个暂缓项和全部剩余声明均保留。

已尝试 OpenClaw 的 Codex autoreview,但工具返回 reviewer_unavailable / engine_failed,未生成报告。独立审查为明确披露的替代审查,不代表 Codex 审查通过。

AI 模型使用说明

  • GPT 6 Astra Fast:实现、验证、核对和 PR 编写;名称来自运行环境。
  • 八个模块审计 agent:只读审计及逐测试记录;运行环境未提供精确模型名称或版本,无法核实。
  • 一个独立覆盖保全审查 agent:删除前质疑及最终差异审查;精确模型名称或版本未提供,无法核实。

本次仅删除测试,不需要更新性能变更记录或运行 GPU sweep。本 PR 按请求保持未合并状态。

中文:删除经覆盖保全审查确认的 36 个重复或自比较测试,保留兼容性与唯一行为覆盖。
@functionstackx
functionstackx requested a review from a team September 29, 2026 05:03
@functionstackx
functionstackx merged commit 6a27af7 into main Sep 29, 2026
4 checks passed
@functionstackx
functionstackx deleted the test/prune-audited-e2e-tests branch September 29, 2026 05:07

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

This is a deletion-only test-maintenance PR (479 lines removed, no production code touched); the one confirmed issue is a minor nit — a stale comment referencing a test function this same PR deletes — not something that should block merging.

Extended reasoning...

The diff only removes test functions and now-unused imports/helpers across 12 test files under inferencex-e2e/infx/tests/; no source, config, or workflow code is touched, so there is no injection/auth/data-exposure surface. I spot-checked the confirmed finding (test_aggregate_power.py:541 comment pointing to test_detect_columns_amd_watch_mode_real_header) and verified via git show against the base commit that this test existed at line 81 before the diff and is indeed removed here, leaving a dangling doc-comment reference — cosmetic only, no behavioral or coverage risk beyond what's already flagged inline. Given the deletion-only nature, the detailed PR description mapping each removed test to retained coverage, and the single confirmed finding being a trivial stale comment, this is approvable without further human deep-dive, though the inline nit is worth a follow-up fix.

Additional findings (outside the current diff — GitHub can't attach inline comments there):

  • 🟡 inferencex-e2e/infx/tests/results/power/test_aggregate_power.py — nit: Maintainers following this comment hit a dead reference: the deleted test it names no longer exists. Line 541's comment reads "measured on MI355X; see test_detect_columns_amd_watch_mode_real_header", but this PR deletes that test function (it covered the AMDSMI 26.2.0 watch-mode header in test_detect_columns_*). Fix: update or drop the "see test_detect_columns_amd_watch_mode_real_header" pointer since the referenced test no longer exists in this file.

    Why this was flagged

    The comment at inferencex-e2e/infx/tests/results/power/test_aggregate_power.py:541 says 'see test_detect_columns_amd_watch_mode_real_header' for the provenance of the MI355X watch-mode CSV header fixture. That test function was one of the ~30 removed in this diff (originally asserting _detect_columns picks socket_power over power_management for that header). No safeguard catches stale comment references; a reader following the pointer to understand where the _MI355X_WATCH_HEADER constant's shape was validated will find nothing, unlike on the base branch where the test exists.

    Verification: nit. The comment at inferencex-e2e/infx/tests/results/power/test_aggregate_power.py:541 reads "# AMDSMI 26.2.0 ... measured on MI355X; see test_detect_columns_amd_watch_mode_real_header)." The diff (git diff dc911f6 HEAD on this file) deletes that test: line 25 of the diff shows "-def test_detect_columns_amd_watch_mode_real_header():". Grep of the whole file confirms the name now appears…

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant