Skip to content

Measure the width of every team BEAM actually creates - #15

Merged
MuteJester merged 2 commits into
mainfrom
parallelism/observe-the-team-that-ran
Nov 16, 2026
Merged

MuteJester merged 2 commits into
mainfrom
parallelism/observe-the-team-that-ran

Conversation

@MuteJester

Copy link
Copy Markdown
Owner

Closes the last gap in the parallelism work: every thread check in this
repository was static, and none could see what the runtime granted.

thread counts proves all 93 parallel regions NAME a thread count.
thread plan wiring proves all 69 GIL-releasing regions RESOLVE a plan.
Neither is evidence about a team that ran -- and that is not hypothetical. The
nesting defect was live in a gate run where thread counts reported OK,
because the clause was present and correct and only its value on a worker
thread was wrong.

What it does

verification/teamwidth.c interposes the two libgomp entry points BEAM creates
teams through (GOMP_parallel and the fused
GOMP_parallel_loop_nonmonotonic_dynamic -- the only two of its ten GOMP
imports that create a team) and records, per region entry, the count requested
and the width that actually ran, split by whether the region itself is nested.

It observes the shipped extension through LD_PRELOAD, with no
recompilation. A check that needed its own build would be checking a different
binary, which is the same reason the gate hashes the artifact rather than
trusting a clean tree.

One entropy() call creates 949 teams. That number is the argument for
measuring at all: reading 93 pragmas cannot tell you what happens 949 times.

What it found immediately

The defect fixed in #14 was latent in every default install, not only under
OMP_MAX_ACTIVE_LEVELS=2 as that PR described it. With the guard removed:

environment requested nested team ran plan
nesting ON (OMP_MAX_ACTIVE_LEVELS=2) 16 16 8
nesting OFF (the default) 16 1 8

In the environment every user actually gets, the defect was invisible in the
width that ran -- libgomp declines to widen a nested region when nesting is
off, so the teams were a blameless 8 and 1 -- while BEAM was still emitting
num_threads(16) on them. The runtime's default policy was the only thing
between the plan and sixteen-way oversubscription. That is luck, not design.

Why both channels are recorded

Because the mutation says they are not interchangeable:

test on the mutant, default env
test_no_region_asks_for_more_than_the_plan FAILS -- "asked for 16 against a plan of 8"
test_no_team_runs_wider_than_the_plan PASSES

An observer recording only the width that ran is blind to this defect in the
default environment. So the requested channel is the primary assertion and the
actual-width tests corroborate it, rather than the other way round.

The tests that stop the file lying

  • test_the_observer_wraps_every_team_creating_entry_point reads the built
    extension's imports and fails if BEAM grows a third way to create a team,
    because the observer would then be blind AND quiet -- the worst failure a
    checking tool has.
  • test_the_measurement_reached_a_nested_region requires depth >= 2, since a
    dead path and a well-behaved one report the same zero. Same lesson as the
    availability estimate in Say whether the availability estimate measured the machine #13, where a quiet runner and a dead reader were
    indistinguishable.
  • test_every_region_that_ran_named_a_thread_count reports what the source scan
    cannot: zero of the 949 regions reached libgomp with nthreads == 0, which is
    what a missing clause looks like from outside. thread counts covers regions
    whether or not they run; this covers regions that ran whether or not the scan
    understood them.

Attribution rests on where the reset goes. LD_PRELOAD interposes libgomp
process-wide, so a counter read at exit would mix in any other OpenMP user -- a
BLAS built against libgomp, say. The child resets immediately before the one
call under test and reads immediately after, with the model built before the
reset, so every region counted is BEAM's.

Second commit

test_the_instrumented_build_enables_avx found the tsan recipe with
^tsan:.*?(?=^\w|\Z), which does not stop at the end of a recipe: adding a
target between tsan and clean grew the match from 10 lines to 26. The test
still passed, but its claim had become "-mavx appears somewhere in this stretch
of the Makefile". Now it matches exactly one recipe, verified by moving -mavx
into the neighbouring target and watching the test fail.

Verification

C suite 454 / 454
Python 1243 passed, 2 skipped
tooling 136 passed, 0 skipped
gate exit 0, every recorded number byte-identical, no inventory line moved
artifacts libbeam.a and the extension hash unchanged

No baseline movement: these tests live in verification/, which the baseline
does not track. The observer is a separate object the shipped build never links.

Runs in CI as a step in the Verification tooling job -- the one job that can,
since LD_PRELOAD over libgomp makes it Linux-only by nature. Without
make teamwidth the tests skip, and a skipped check is not a check.

`test_the_instrumented_build_enables_avx` found the tsan recipe with
`^tsan:.*?(?=^\w|\Z)` -- everything up to the next line starting with a word
character. Make recipes are tab-indented and the comments between targets start
with `#`, so that pattern does not stop at the end of the recipe: it runs
through any following comment block and through a following rule whose target
begins with `$(`, stopping only at the next target named by a bare word.

Found while adding a target between `tsan` and `clean`, which grew the match
from 10 lines to 26. The test still passed, but its claim had quietly become
"-mavx appears somewhere in this stretch of the Makefile" rather than "the tsan
recipe passes -mavx", and it would have gone on passing with the flag moved
into a neighbouring recipe and out of this one.

Now it matches the target line plus the tab-indented lines under it, which is
exactly one recipe. The continuation-collapsing step above it runs first, so a
backslash-continued line is already one line by the time this applies.

Verified by moving `-mavx` out of the tsan recipe and into the neighbouring
one: the test fails, where before the change it passed.
Every thread check in this repository was static. `thread counts` proves all 93
parallel regions NAME a thread count; `thread plan wiring` proves all 69
GIL-releasing regions RESOLVE a plan. Neither can see what the runtime granted,
and that gap is not hypothetical: the nesting defect was live in a gate run
where `thread counts` reported OK, because the clause was present and correct
and only its value on a worker thread was wrong.

verification/teamwidth.c interposes the two libgomp entry points BEAM creates
teams through -- GOMP_parallel and the fused
GOMP_parallel_loop_nonmonotonic_dynamic, the only two of its ten GOMP imports
that create a team -- and records, per region entry, the count requested and
the width that actually ran, split by whether the region itself is nested.

It observes the SHIPPED extension through LD_PRELOAD, with no recompilation. A
check that needed its own build would be checking a different binary, which is
the same reason the gate hashes the artifact rather than trusting a clean tree.

One entropy() call creates 949 teams. That number is the argument for measuring
at all: reading 93 pragmas cannot tell you what happens 949 times, and a region
inside a loop is one pragma and ten thousand entries.

BOTH channels are recorded, and the mutation is why. With the nested-region
guard removed from beam_threads():

  nesting ON  (OMP_MAX_ACTIVE_LEVELS=2)   requested 16, nested ran 16, plan 8
  nesting OFF (the default)               requested 16, nested ran  1, plan 8

So in the environment every user actually gets, the defect was INVISIBLE in the
width that ran -- libgomp declines to widen a nested region when nesting is off,
so the teams were a blameless 8 and 1 -- while BEAM was still emitting
num_threads(16) on them. The runtime's default policy was the only thing
between the plan and sixteen-way oversubscription, which is luck rather than
design. A check on the width that ran PASSES on that mutant;
test_no_region_asks_for_more_than_the_plan fails it with no special environment
at all, and is therefore the primary assertion rather than a second opinion.

Two tests exist to stop the file lying. One reads the built extension's imports
and fails if BEAM grows a third way to create a team, because the observer
would then be blind AND quiet -- the worst failure a checking tool has. The
other requires the measurement to have reached depth >= 2, since a dead path
and a well-behaved one report the same zero; this is the same lesson as the
availability estimate, where a quiet runner and a dead reader were
indistinguishable.

A third reports what the source scan cannot: zero of the 949 regions reached
libgomp with nthreads == 0, which is what a missing num_threads clause looks
like from the outside. `thread counts` covers regions whether or not they run;
this covers regions that ran whether or not the scan understood them.

Attribution rests on where the reset goes. LD_PRELOAD interposes libgomp
process-wide, so a counter read at exit would mix in any other OpenMP user --
a BLAS built against libgomp, say. The child resets immediately before the one
call under test and reads immediately after, with the model built before the
reset, so every region counted is BEAM's.

Runs in CI as a step in the Verification tooling job, which is the one job that
can: it interposes libgomp through LD_PRELOAD, so it is Linux-only by nature.
Without `make teamwidth` the tests skip, and a skipped check is not a check.

Verified: C 454 tests passing, Python 1243 passed and 2 skipped, tooling 136
passed and 0 skipped, gate exit 0 with every recorded number byte-identical and
no inventory line moved -- these tests live in verification/, which the
baseline does not track. Both artifacts hash unchanged, the observer being a
separate object that the shipped build never links.
@MuteJester
MuteJester merged commit 3aed6dd into main Nov 16, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant