Skip to content

Ecosystem runner silently skips 3 of 19 repos — build systems below the root are never detected #17

Description

@srpatcha

detect_kinds() inspects only the repository root. Three of the nineteen repos
in the workspace keep their build systems one or more levels down, so the
ecosystem runner reports them as unknown, finds no runner, and skips them.

  discovered: 19
   eCAD-Hardware-Products   unknown
   eos-aero                 unknown
   eos-health               unknown
   ... 16 others detected correctly

All three do have build systems:

repo build file depth
eCAD-Hardware-Products tools/requirements.txt 2
eos-health firmware/build-system/CMakeLists.txt 3
eos-health apps/web/package.json 3
eos-aero AeroSwift/software/web_app/package.json 4

So 16% of the organisation is invisible to the tool whose purpose is to test the
organisation. eos-health is the sharpest case — it ships firmware with a CMake
build and a web app, and neither is ever compiled or tested by the runner.

Why it reads as fine

The summary line counts these as skipped:

Repos:  19 discovered | N passed | 0 failed | 3 skipped

0 failed with everything green is the same shape a healthy run produces. A
skip is indistinguishable from a pass at a glance, and the reason string
(no runner for a 'unknown' project) only appears in the detail rows.

This is the same class as the fabricated-pass bug removed in #16 — a repo that
was never really tested reporting as though nothing was wrong — arriving through
a different door.

Constraint on the fix

The runners take (name, path) and build from path. Returning cmake for
eos-health without also carrying where would make the runner invoke
cmake -S <repo root>, which has no CMakeLists.txt. It would turn a silent
skip into a spurious failure, which is not an improvement.

The detection has to yield the component's directory alongside its kind.

Proposed change

Add detect_components(path) returning (kind, directory) pairs, and have
test_repo_all iterate those, passing each component's own directory to the
runner and labelling the result with the subdirectory so the output stays
readable when one repo yields several components.

Root detection runs first and unchanged; the nested scan happens only when the
root yields nothing. That keeps the 16 currently-detected repos on exactly the
path they take today, so this cannot regress them.

The nested scan needs a bounded depth and must not descend into node_modules,
.git, build, dist, venv or .venv — a vendored package.json under
node_modules would otherwise be picked up as a component.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions