[scb_to_harbor] Node image template broken by unpinned npm@latest; base image missing rabbitmq-server/redis-server
Two environment issues in the Harbor conversion that break or skew results.
Both observed on scb-problems main@ef6a9dd (latest as of 2026-09-01).
A. npm install -g npm@latest makes 5 node-based problems unbuildable
scripts/scb_to_harbor_lib/templates.py:89 (node image template) pins
NODE_VERSION=22.12.0 but then floats npm:
RUN ... nvm install "$NODE_VERSION" && ... && npm install -g npm@latest && ...
npm 12 raised its engine floor to node ^22.22.2 || ^24.15.0 || >=26.0.0, so
any fresh build of these images now fails:
npm error code EBADENGINE
npm error notsup Not compatible with your version of node/npm: [email protected]
npm error notsup Required: {"node":"^22.22.2 || ^24.15.0 || >=26.0.0"}
npm error notsup Actual: {"npm":"10.9.0","node":"v22.12.0"}
#6 ERROR: process "/bin/bash -lc mkdir -p \"$NVM_DIR\" ... npm install -g npm@latest ..." did not complete successfully: exit code: 1
Affected problems (the only ones using the node template): code_search,
database_migration, dynamic_buffer, l2m, test_translator. On Harbor these
fail during environment build, so the containers score 0 with zero agent
steps — the problems are effectively un-runnable today unless the image
happens to be cached from before the npm 12 release.
Fix (either): pin npm to a compatible major (npm install -g npm@11; npm 11
accepts node 22.12.0), or bump NODE_VERSION to >=22.22.2 (or 24.x).
Pinning npm is the safer choice — floating npm@latest will keep
reintroducing this class of failure.
B. Base image lacks rabbitmq-server / redis-server; mocked_http tests silently skip
The native runner's base image
(slop-code-bench/execution/docker_runtime/setup_base.docker.j2) ships
rabbitmq-server, redis-server, minio, etc. The Harbor image template
(templates.py) installs only basic apt packages, so service-dependent tests
in mocked_http can never run:
tests/conftest.py skips when the rabbitmq-server / redis-server
binaries are missing (pytest.skip),
test_checkpoint_3.py::test_external_redis_mode (1 case) and 5
redis/rabbitmq persistence cases in test_checkpoint_5.py therefore always
skip — and because prior tests re-run as regression, the skips recur at
every later checkpoint (ck4–ck8).
Skipped tests drop out of the pass-rate denominator, so agents that never
implement external redis/rabbitmq persistence are not penalized on Harbor
while they are on the native runner — a systematic score inflation for this
problem.
Fix: add rabbitmq-server and redis-server to the mocked_http image (via a
conversion.toml override or the shared template). A quick audit suggests no
other problem's tests depend on missing binaries, but it may be worth
diffing the native base image against the Harbor template once to be sure
(minio, protobuf-compiler, default-jre, graphviz are also absent).
Environment
- scb-problems
main@ef6a9dd; Harbor registry dataset
gabeorlanski/slopcodebench is generated from the same templates
- Build failures and skips observed 2026-08-31 with npm 12.0.2 latest
[scb_to_harbor] Node image template broken by unpinned npm@latest; base image missing rabbitmq-server/redis-server
Two environment issues in the Harbor conversion that break or skew results.
Both observed on scb-problems
main@ef6a9dd(latest as of 2026-09-01).A.
npm install -g npm@latestmakes 5 node-based problems unbuildablescripts/scb_to_harbor_lib/templates.py:89(node image template) pinsNODE_VERSION=22.12.0but then floats npm:npm 12 raised its engine floor to
node ^22.22.2 || ^24.15.0 || >=26.0.0, soany fresh build of these images now fails:
Affected problems (the only ones using the node template): code_search,
database_migration, dynamic_buffer, l2m, test_translator. On Harbor these
fail during environment build, so the containers score 0 with zero agent
steps — the problems are effectively un-runnable today unless the image
happens to be cached from before the npm 12 release.
Fix (either): pin npm to a compatible major (
npm install -g npm@11; npm 11accepts node 22.12.0), or bump
NODE_VERSIONto>=22.22.2(or 24.x).Pinning npm is the safer choice — floating
npm@latestwill keepreintroducing this class of failure.
B. Base image lacks rabbitmq-server / redis-server; mocked_http tests silently skip
The native runner's base image
(
slop-code-bench/execution/docker_runtime/setup_base.docker.j2) shipsrabbitmq-server, redis-server, minio, etc. The Harbor image template
(
templates.py) installs only basic apt packages, so service-dependent testsin mocked_http can never run:
tests/conftest.pyskips when therabbitmq-server/redis-serverbinaries are missing (
pytest.skip),test_checkpoint_3.py::test_external_redis_mode(1 case) and 5redis/rabbitmq persistence cases in
test_checkpoint_5.pytherefore alwaysskip — and because prior tests re-run as regression, the skips recur at
every later checkpoint (ck4–ck8).
Skipped tests drop out of the pass-rate denominator, so agents that never
implement external redis/rabbitmq persistence are not penalized on Harbor
while they are on the native runner — a systematic score inflation for this
problem.
Fix: add
rabbitmq-serverandredis-serverto the mocked_http image (via aconversion.tomloverride or the shared template). A quick audit suggests noother problem's tests depend on missing binaries, but it may be worth
diffing the native base image against the Harbor template once to be sure
(minio, protobuf-compiler, default-jre, graphviz are also absent).
Environment
main@ef6a9dd; Harbor registry datasetgabeorlanski/slopcodebenchis generated from the same templates