You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
benchmarks: run_public_baseline.sh can't regenerate from a clean checkout (missing -static wrapper crates → 'Could not find libperry_runtime.a') #7012
./benchmarks/run_public_baseline.sh cannot regenerate the public artifact from a clean checkout: every perry compile in the measurement legs dies with Could not find libperry_runtime.a.
Cause
The script builds only:
cargo build --release -p perry-runtime -p perry-stdlib -p perry
but perry-runtime / perry-stdlib are rlib-only — libperry_runtime.a and libperry_stdlib.a come from the separate perry-runtime-static / perry-stdlib-static wrapper crates. So target/release/ ends up with the perry binary but no static archives, and the compiler can't link anything it compiles.
This is invisible on a long-lived working copy (the maintainer's tree already has the archives from unrelated builds), which is why it has never been hit before. It reproduces reliably on a fresh git worktree / clone.
Why it's worth fixing rather than documenting
The failure lands at the worst possible moment. The script's shape is:
cargo build --release …
wait_for_quiet (needs CPU ≤25% for 60s, hard-aborts after 15 min)
So on a contended machine you wait hours for an idle window, the run finally starts, and then dies in the first measurement leg — and you wait for the next window to find out whether the fix worked. Total cost of the missing -static is measured in hours, not seconds.
Optionally add a fail-fast assertion after the build ([ -f target/release/libperry_runtime.a ] || fail "…") so a future packaging change surfaces immediately instead of during measurement.
Sequencing caveat for whoever takes this
run_public_baseline.sh is itself listed in HARNESS_PATHS, so editing it changes harness_fingerprint and invalidates any existing artifact (validate_public → "harness changed; regenerate"). The script fix and an artifact regeneration therefore have to land together — the fix cannot be shipped in the same PR as an artifact that was generated with the old script, and it can't be shipped alone without immediately reddening the lint freshness gate.
Found while regenerating the baseline to clear the lint gate after the workspace-restructuring merges (#6758/#6761) drifted source_fingerprint via Cargo.toml.
Second fragility found while working around this: HARNESS_PATHS includes the whole benchmarks/honest_bench directory, which contains npm lockfiles (workloads/4_http_fastify/node/package-lock.json). So a dependabot bump can silently invalidate the public artifact — #6752/#6753 changed 3 lines in that lockfile, which is enough to shift harness_fingerprint and make a freshly-generated artifact fail validate_public with "harness changed; regenerate".
That turns baseline regeneration into a race against dependabot: the artifact has to be generated from a tree whose fingerprinted files match main at merge time, and a lockfile bump landing mid-run invalidates ~2 hours of measurement (5 quiet-CPU gates). It cost me one full window today — I had pinned a worktree, and the bump landed while I was waiting for an idle machine.
Worth considering whether the honest_bench entry should fingerprint only the harness scripts + results/expected.json rather than the entire subtree, so vendored lockfiles don't participate. That is orthogonal to the -static fix above but hits the same workflow.
Activity
proggeramlug commentedon Jul 29, 2026
Second fragility found while working around this:
HARNESS_PATHSincludes the wholebenchmarks/honest_benchdirectory, which contains npm lockfiles (workloads/4_http_fastify/node/package-lock.json). So a dependabot bump can silently invalidate the public artifact — #6752/#6753 changed 3 lines in that lockfile, which is enough to shiftharness_fingerprintand make a freshly-generated artifact failvalidate_publicwith "harness changed; regenerate".That turns baseline regeneration into a race against dependabot: the artifact has to be generated from a tree whose fingerprinted files match main at merge time, and a lockfile bump landing mid-run invalidates ~2 hours of measurement (5 quiet-CPU gates). It cost me one full window today — I had pinned a worktree, and the bump landed while I was waiting for an idle machine.
Worth considering whether the honest_bench entry should fingerprint only the harness scripts +
results/expected.jsonrather than the entire subtree, so vendored lockfiles don't participate. That is orthogonal to the-staticfix above but hits the same workflow.fix(benchmarks): public baseline must build the -static wrapper crates (