Skip to content

[scb_to_harbor] Node image template broken by unpinned npm@latest; base image missing rabbitmq-server/redis-server #31

Description

@cvbhjkl

[scb_to_harbor] Node image template broken by unpinned npm@latest; base image missing rabbitmq-server/redis-server

Two environment issues in the Harbor conversion that break or skew results.
Both observed on scb-problems main@ef6a9dd (latest as of 2026-09-01).

A. npm install -g npm@latest makes 5 node-based problems unbuildable

scripts/scb_to_harbor_lib/templates.py:89 (node image template) pins
NODE_VERSION=22.12.0 but then floats npm:

RUN ... nvm install "$NODE_VERSION" && ... && npm install -g npm@latest && ...

npm 12 raised its engine floor to node ^22.22.2 || ^24.15.0 || >=26.0.0, so
any fresh build of these images now fails:

npm error code EBADENGINE
npm error notsup Not compatible with your version of node/npm: [email protected]
npm error notsup Required: {"node":"^22.22.2 || ^24.15.0 || >=26.0.0"}
npm error notsup Actual:   {"npm":"10.9.0","node":"v22.12.0"}
#6 ERROR: process "/bin/bash -lc mkdir -p \"$NVM_DIR\" ... npm install -g npm@latest ..." did not complete successfully: exit code: 1

Affected problems (the only ones using the node template): code_search,
database_migration, dynamic_buffer, l2m, test_translator
. On Harbor these
fail during environment build, so the containers score 0 with zero agent
steps — the problems are effectively un-runnable today unless the image
happens to be cached from before the npm 12 release.

Fix (either): pin npm to a compatible major (npm install -g npm@11; npm 11
accepts node 22.12.0), or bump NODE_VERSION to >=22.22.2 (or 24.x).
Pinning npm is the safer choice — floating npm@latest will keep
reintroducing this class of failure.

B. Base image lacks rabbitmq-server / redis-server; mocked_http tests silently skip

The native runner's base image
(slop-code-bench/execution/docker_runtime/setup_base.docker.j2) ships
rabbitmq-server, redis-server, minio, etc. The Harbor image template
(templates.py) installs only basic apt packages, so service-dependent tests
in mocked_http can never run:

  • tests/conftest.py skips when the rabbitmq-server / redis-server
    binaries are missing (pytest.skip),
  • test_checkpoint_3.py::test_external_redis_mode (1 case) and 5
    redis/rabbitmq persistence cases in test_checkpoint_5.py therefore always
    skip — and because prior tests re-run as regression, the skips recur at
    every later checkpoint (ck4–ck8).

Skipped tests drop out of the pass-rate denominator, so agents that never
implement external redis/rabbitmq persistence are not penalized on Harbor
while they are on the native runner — a systematic score inflation for this
problem.

Fix: add rabbitmq-server and redis-server to the mocked_http image (via a
conversion.toml override or the shared template). A quick audit suggests no
other problem's tests depend on missing binaries, but it may be worth
diffing the native base image against the Harbor template once to be sure
(minio, protobuf-compiler, default-jre, graphviz are also absent).

Environment

  • scb-problems main@ef6a9dd; Harbor registry dataset
    gabeorlanski/slopcodebench is generated from the same templates
  • Build failures and skips observed 2026-08-31 with npm 12.0.2 latest

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions