From 80cb0f0a8effd4eb12fc4c4820bcc3e050d97219 Mon Sep 17 00:00:00 2001 From: thomas Date: Sun, 15 Nov 2026 11:08:00 +0200 Subject: [PATCH] Sense contention on the platforms without /proc beam_available_cores() read /proc/loadavg inside #if defined(__linux__) and nothing else, so on macOS and Windows -- two of the three platforms BEAM ships wheels for -- the runnable count was always unknown. Unknown means "assume no contention", so clause 3 planned eight threads whatever the machine was doing. Phase 0's catastrophe, 301x slower than serial on a saturated node, was defended against on one platform of three. The obvious portable answer is already ruled out by the measurements in avail.h: the one-minute load average is wrong by -6 at eight competitors, and half a second after twelve spinners exited it still read 9.22 while the runnable count had returned to 2. A plan built on it "would serialise every call for a minute after BEAM's own previous call finished". Both platforms have getloadavg(). It is not the answer. So they difference two cumulative CPU-time readings instead: competitors = ceil((system_busy_ns - own_ns) / wall_ns), clamped macOS reads host_statistics(HOST_CPU_LOAD_INFO), getrusage(RUSAGE_SELF) and clock_gettime(CLOCK_MONOTONIC); Windows reads GetSystemTimes, GetProcessTimes and the performance counter. One syscall each and no sleep, which the cost budget demands: this runs at every entry point with no caching, and Linux answers in microseconds. own_ns is subtracted, and that is the whole design. Without it a window spanning BEAM's own eight-thread region reads as eight cores busy, so every call would narrow the next BECAUSE OF THE PREVIOUS ONE -- the same feedback loop the load average was rejected for. Linux escapes it for free, the runnable count being instantaneous: by the time it reads, the workers have exited. A sampled estimate has to subtract its own usage to measure the same quantity. The three state rules live in beam_competitors_from_samples rather than in platform code, so they are testable on any machine. A thread's first reading is only a baseline. A window below twenty milliseconds reports unknown and leaves the baseline to age -- replacing it per call is the bug that would make the mechanism useless, since a tight loop of short calls would restart the window and never reach twenty milliseconds, on exactly the workload that needs it. A reading that moves backwards resets rather than subtracting into an unsigned underflow. Every failure path returns -1, which is what these platforms returned on every call before, so this cannot be worse than what it replaced; it can only start being right sooner. It is second best and says so: a window rather than an instant, so it lags, and the lag lands on the safe side. Planning narrow costs about 2x and planning wide cost up to 3814x, which is also why a fraction of a core rounds up. Eleven tests, four of them mutated against the real suite: advancing the baseline on a short window, dropping the backwards guard, dropping the subtraction of our own CPU time, and rounding down instead of up each fail exactly the test that names the property. Two defects in the first draft, found by re-reading rather than by a test, and neither reachable from Linux. mach_host_self() hands out a send right per call and was leaking one Mach port reference per entry point. And counter * 1000000000 on the performance counter overflows int64 after about fifteen minutes of uptime at 10 MHz -- longer than a CI runner lives, so it would have passed green here and returned nonsense on a machine up since Tuesday. Two stale claims corrected on the way. avail.h credited "the policy's own halving" with the conservative fallback; plan.c stopped halving when the ceiling form replaced it, and the ceiling does that work now. And test_avail.c said "no /proc, as on Windows" while describing the unknown path as not a special case anywhere -- it was the gap, in a comment. Verified: C 450 passed, Python 1241 passed and 2 skipped, gate exit 0 with no FAIL and no note lines, every recorded number byte-identical, extension rebuilt from a2cc902b to 334d37e9. The macOS and Windows readers do not compile on this machine; CI is their first build and their first run. --- docs/src/content/docs/reference/runtime.md | 22 +- src/native/core/avail.c | 190 ++++++++ src/native/internal/beam/avail.h | 93 +++- tests/native/test_avail.c | 188 +++++++- verification/baseline/RERECORDED.md | 55 +++ verification/baseline/c_cases.availsample.txt | 439 ++++++++++++++++++ verification/baseline/c_cases.txt | 11 + .../baseline/c_symbols.availsample.txt | 261 +++++++++++ verification/baseline/c_symbols.txt | 2 + 9 files changed, 1249 insertions(+), 12 deletions(-) create mode 100644 verification/baseline/c_cases.availsample.txt create mode 100644 verification/baseline/c_symbols.availsample.txt diff --git a/docs/src/content/docs/reference/runtime.md b/docs/src/content/docs/reference/runtime.md index 27735ca..db87f7b 100644 --- a/docs/src/content/docs/reference/runtime.md +++ b/docs/src/content/docs/reference/runtime.md @@ -15,10 +15,24 @@ BEAM.threads() # the limit in force (0: the system's choice) BEAM.effective_threads() # what a parallel step would start right now ``` -BEAM uses all available cores by default, or `OMP_NUM_THREADS` if it is set. `set_threads` is -a hard limit, which is what you want inside a job scheduler or next to another parallel step. It -applies to every model in the process; `set_threads(0)` restores the default. Set it before -starting work, not while another thread is computing. +By default BEAM picks the width itself, from how busy the machine is and from where its own +work stops scaling — on a quiet workstation that is up to eight threads, and on a shared node +with little free it is far fewer, which is the difference between a job that runs and one that +thrashes. `OMP_NUM_THREADS`, if you set it, decides instead: BEAM treats it as your instruction +and stands aside. + +`set_threads` is a hard limit, which is what you want inside a job scheduler or next to another +parallel step. It applies to every model in the process; `set_threads(0)` restores the default. +Set it before starting work, not while another thread is computing. + +```python +BEAM.parallel_plan() # what the default resolved to, and from what +``` + +For a benchmark or a figure you want to reproduce exactly, pin the machine estimate with the +`BEAM_AVAILABLE_CORES` environment variable; the plan then depends only on your own settings +rather than on what else happened to be running. `parallel_plan()` reports the width and the +inputs it came from, so a surprising thread count can be explained rather than guessed at. **Results do not depend on the number of threads.** Every sum is taken in a fixed order, so one thread and sixteen give the same answer, bit for bit. diff --git a/src/native/core/avail.c b/src/native/core/avail.c index 149fa64..0a59a41 100644 --- a/src/native/core/avail.c +++ b/src/native/core/avail.c @@ -25,6 +25,27 @@ #include #endif +/* + * The sampled estimate, for the platforms with no /proc. See avail.h for why + * the quantity is competitors rather than load, and why that costs a second + * clock read. + */ +#if defined(__APPLE__) +#include +#include +#include +#include +#include +#elif defined(_WIN32) +#include +#endif + +#ifdef _MSC_VER + #define TLS __declspec(thread) +#else + #define TLS _Thread_local +#endif + /* ── Parsers ─────────────────────────────────────────────────── */ int beam_parse_nr_running(const char *loadavg_text) { @@ -92,6 +113,44 @@ int beam_available_from(int allowed, int nr_running) { return avail < 1 ? 1 : avail; } +int beam_competitors_from_cpu_time(uint64_t sys_busy_ns, uint64_t own_ns, + uint64_t wall_ns, int allowed) { + if (allowed < 1) allowed = 1; + if (wall_ns < BEAM_AVAIL_MIN_WINDOW_NS) return -1; + /* + * Our own time can exceed the system total across a non-atomic pair of + * reads. Equal means the machine ran nothing but us. + */ + if (sys_busy_ns <= own_ns) return 1; + const uint64_t other = sys_busy_ns - own_ns; + /* Ceiling: half a core of competition is one competitor, not none. */ + uint64_t cores = (other + wall_ns - 1u) / wall_ns; + if (cores > (uint64_t)allowed) cores = (uint64_t)allowed; + return (int)cores + 1; +} + +int beam_competitors_from_samples(BeamAvailSample *prev, uint64_t sys, + uint64_t own, uint64_t wall, int allowed) { + if (!prev) return -1; + + if (!prev->have) { /* rule 1 */ + prev->sys = sys; prev->own = own; prev->wall = wall; prev->have = 1; + return -1; + } + if (wall <= prev->wall || sys < prev->sys || own < prev->own) { /* rule 3 */ + prev->sys = sys; prev->own = own; prev->wall = wall; + return -1; + } + const uint64_t window = wall - prev->wall; + if (window < BEAM_AVAIL_MIN_WINDOW_NS) return -1; /* rule 2 */ + + const int nr = beam_competitors_from_cpu_time(sys - prev->sys, + own - prev->own, + window, allowed); + prev->sys = sys; prev->own = own; prev->wall = wall; + return nr; +} + /* ── Reading the machine ─────────────────────────────────────── */ int beam_cores_present(void) { @@ -146,6 +205,129 @@ int beam_allowed_cores(void) { return allowed; } +#if defined(__APPLE__) || defined(_WIN32) +/* + * Busy CPU time across every CPU, our own process's CPU time, and a monotonic + * wall clock -- all in nanoseconds, so the pure arithmetic above never sees a + * platform's tick unit. Each returns 0 on failure, and a failed read means + * "unknown", which is the existing safe fallback rather than a special case. + */ +static int read_system_busy_ns(uint64_t *out) { +#if defined(__APPLE__) + host_cpu_load_info_data_t info; + mach_msg_type_number_t count = HOST_CPU_LOAD_INFO_COUNT; + /* + * mach_host_self() hands out a send right, and this runs at every entry + * point: without the deallocate it leaks one Mach port reference per BEAM + * call, which no test would notice and a long-lived process would. + */ + const host_t host = mach_host_self(); + const kern_return_t kr = host_statistics(host, HOST_CPU_LOAD_INFO, + (host_info_t)&info, &count); + mach_port_deallocate(mach_task_self(), host); + if (kr != KERN_SUCCESS) return 0; + /* HOST_CPU_LOAD_INFO is already summed over the CPUs. Idle is excluded. */ + const uint64_t ticks = (uint64_t)info.cpu_ticks[CPU_STATE_USER] + + (uint64_t)info.cpu_ticks[CPU_STATE_SYSTEM] + + (uint64_t)info.cpu_ticks[CPU_STATE_NICE]; + const long hz = sysconf(_SC_CLK_TCK); + if (hz <= 0) return 0; + *out = ticks * (1000000000ull / (uint64_t)hz); + return 1; +#else + FILETIME idle, kernel, user; + if (!GetSystemTimes(&idle, &kernel, &user)) return 0; + ULARGE_INTEGER i, k, u; + i.LowPart = idle.dwLowDateTime; i.HighPart = idle.dwHighDateTime; + k.LowPart = kernel.dwLowDateTime; k.HighPart = kernel.dwHighDateTime; + u.LowPart = user.dwLowDateTime; u.HighPart = user.dwHighDateTime; + /* Kernel time INCLUDES idle, so busy is kernel + user - idle. */ + const uint64_t total = (k.QuadPart + u.QuadPart) * 100ull; /* 100-ns units */ + const uint64_t spare = i.QuadPart * 100ull; + *out = total > spare ? total - spare : 0ull; + return 1; +#endif +} + +static int read_own_cpu_ns(uint64_t *out) { +#if defined(__APPLE__) + struct rusage ru; + if (getrusage(RUSAGE_SELF, &ru) != 0) return 0; + *out = (uint64_t)ru.ru_utime.tv_sec * 1000000000ull + + (uint64_t)ru.ru_utime.tv_usec * 1000ull + + (uint64_t)ru.ru_stime.tv_sec * 1000000000ull + + (uint64_t)ru.ru_stime.tv_usec * 1000ull; + return 1; +#else + FILETIME created, exited, kernel, user; + if (!GetProcessTimes(GetCurrentProcess(), &created, &exited, + &kernel, &user)) return 0; + ULARGE_INTEGER k, u; + k.LowPart = kernel.dwLowDateTime; k.HighPart = kernel.dwHighDateTime; + u.LowPart = user.dwLowDateTime; u.HighPart = user.dwHighDateTime; + *out = (k.QuadPart + u.QuadPart) * 100ull; + return 1; +#endif +} + +static int read_wall_ns(uint64_t *out) { +#if defined(__APPLE__) + struct timespec ts; + if (clock_gettime(CLOCK_MONOTONIC, &ts) != 0) return 0; + *out = (uint64_t)ts.tv_sec * 1000000000ull + (uint64_t)ts.tv_nsec; + return 1; +#else + LARGE_INTEGER freq, now; + if (!QueryPerformanceFrequency(&freq) || freq.QuadPart <= 0) return 0; + if (!QueryPerformanceCounter(&now) || now.QuadPart < 0) return 0; + const uint64_t q = (uint64_t)now.QuadPart, f = (uint64_t)freq.QuadPart; + /* + * Seconds and remainder separately. Counter times a billion overflows + * int64 after roughly fifteen minutes of uptime at the usual 10 MHz -- + * longer than a CI runner lives, so the obvious one-liner passes here and + * returns nonsense on a machine that has been up since Tuesday. + */ + *out = (q / f) * 1000000000ull + ((q % f) * 1000000000ull) / f; + return 1; +#endif +} + +/* + * The previous sample, per thread. + * + * Thread-local for the reason threads.c gives for the plan itself: two BEAM + * calls on two threads sharing one baseline would measure each other's window + * and neither would get a window it could interpret. A thread's first call + * therefore reports unknown, which is what this platform did on every call + * before. + * + * A window shorter than the accounting resolution does NOT replace the + * baseline. Overwriting it on every call is the bug that makes this whole + * mechanism useless: a tight loop of short calls would reset the start of the + * window each time and never accumulate twenty milliseconds, so the estimate + * would be unknown forever on exactly the workload that needs it. The + * baseline is left to age instead. + */ +/* + * The previous sample, per thread. + * + * Thread-local for the reason threads.c gives for the plan itself: two BEAM + * calls on two threads sharing one baseline would measure each other's window + * and neither would get a window it could interpret. A thread's first call + * therefore reports unknown, which is what this platform answered on EVERY + * call before, so nothing here can be worse than it was. + */ +static TLS BeamAvailSample tl_sample; + +static int sampled_competitors(int allowed) { + uint64_t sys = 0u, own = 0u, wall = 0u; + if (!read_system_busy_ns(&sys)) return -1; + if (!read_own_cpu_ns(&own)) return -1; + if (!read_wall_ns(&wall)) return -1; + return beam_competitors_from_samples(&tl_sample, sys, own, wall, allowed); +} +#endif /* __APPLE__ || _WIN32 */ + /* * An explicit pin, for reproducibility. * @@ -180,6 +362,14 @@ int beam_available_cores(void) { if (read_small_file("/proc/loadavg", buf, sizeof buf)) { nr = beam_parse_nr_running(buf); } +#elif defined(__APPLE__) || defined(_WIN32) + /* + * No /proc, so no instantaneous runnable count. The sampled estimate is + * second best and says so: it lags, and it reports unknown until a thread + * has a window worth measuring. Unknown is what this platform answered on + * every call before, so nothing here can be worse than it was. + */ + nr = sampled_competitors(allowed); #endif return beam_available_from(allowed, nr); } diff --git a/src/native/internal/beam/avail.h b/src/native/internal/beam/avail.h index 4d44441..8d34969 100644 --- a/src/native/internal/beam/avail.h +++ b/src/native/internal/beam/avail.h @@ -38,14 +38,30 @@ * conservative is the cheap direction -- planning too narrow costs about 2x, * planning too wide cost up to 3814x. * - * Every parser returns -1 for "could not tell" rather than guessing. -1 then - * means "assume no contention", and the policy's own halving supplies the - * conservative answer. That is also exactly what a machine with no /proc at - * all gets, so the non-Linux fallback is not a special case anywhere. + * Every reader returns -1 for "could not tell" rather than guessing, and -1 + * means "assume no contention". It is the one answer that cannot be wrong in + * the expensive direction by more than the ceiling allows: clause 3 of the + * policy is min(available, 8), so an unknown machine is planned at 8 and the + * error is bounded by that, not by the core count. (An earlier version of + * plan.c halved availability instead, and this paragraph used to credit the + * halving with the conservative answer; the halving is gone and the ceiling + * does that work now.) + * + * What the platforms without /proc do. They have no instantaneous runnable + * count, and the load average they DO have is ruled out above. So they + * difference two cumulative CPU-time readings instead -- see + * beam_competitors_from_cpu_time and beam_competitors_from_samples -- which + * measures competitors over a window rather than at an instant. It is second + * best, and the second-best-ness is explicit: it lags, it reports unknown + * until a thread has a window worth measuring, and every failure path lands + * on the same -1 those platforms used to return on every call. It cannot be + * worse than what it replaced; it can only start being right sooner. */ #ifndef BEAM_INTERNAL_AVAIL_H #define BEAM_INTERNAL_AVAIL_H +#include + /** * Runnable scheduling entities, from the fourth field of /proc/loadavg * ("running/total"). -1 when the text cannot be read. @@ -89,6 +105,75 @@ int beam_allowed_cores(void); */ int beam_available_from(int allowed, int nr_running); +/** + * Minimum window worth measuring, in nanoseconds. CPU time is accounted in + * ticks of roughly 10 ms on both platforms that need the sampled estimate, so + * a shorter window carries no information: the deltas are zero or one tick + * and the quotient is noise. + */ +#define BEAM_AVAIL_MIN_WINDOW_NS 20000000u + +/** + * Competitors implied by two cumulative CPU-time samples, in the units + * `beam_available_from` takes: runnable entities including the caller. -1 when + * the window is too short to mean anything. + * + * This is how a machine with no /proc answers the question Linux answers with + * the runnable count. The quantity is competitors, NOT total load, which is + * why `own_ns` is subtracted: a window spanning BEAM's own eight-thread region + * reads as eight cores busy, and counting that as contention would narrow the + * next call BECAUSE OF THE PREVIOUS ONE -- the feedback loop this file rejects + * the load average for. Linux escapes it for free, the runnable count being + * instantaneous: by the time it reads, BEAM's workers have exited. + * + * Both reads are averages over the window rather than instants, so this signal + * lags where the runnable count does not. The lag is bounded and lands on the + * safe side: a machine that was busy and is now idle plans narrow, costing + * about 2x, while the error in the other direction cost up to 3814x. A + * fraction of a core therefore rounds UP, and a sample whose own time exceeds + * the system total -- the two clocks are not read atomically -- reports no + * competition rather than negative competition. + * + * Pure, so the arithmetic is tested directly; the platform readers that + * produce the samples are the only part a test cannot pin to an exact answer. + */ +int beam_competitors_from_cpu_time(uint64_t sys_busy_ns, uint64_t own_ns, + uint64_t wall_ns, int allowed); + +/** + * One thread's previous CPU-time sample. Exposed so that the sample-to-sample + * step below can be driven from a test rather than from two clocks. + */ +typedef struct { + uint64_t sys; /**< Busy CPU nanoseconds across every CPU. */ + uint64_t own; /**< This process's CPU nanoseconds. */ + uint64_t wall; /**< Monotonic nanoseconds. */ + int have; /**< Zero until the first reading is stored. */ +} BeamAvailSample; + +/** + * The estimate from a new reading and the previous sample, updating the sample + * in place. -1 when the pair cannot be interpreted yet. + * + * Pure apart from `*prev`, which is the whole point: the three rules that make + * the sampled estimate work are here rather than in platform code, so a test + * can pin them on any machine. + * + * 1. A thread's FIRST reading becomes the baseline and reports unknown. + * There is nothing to difference against. + * 2. A window below `BEAM_AVAIL_MIN_WINDOW_NS` reports unknown and LEAVES + * THE BASELINE ALONE, so it keeps ageing. Replacing it on every call is + * the bug that would make the mechanism useless: a tight loop of short + * calls would restart the window each time and never reach twenty + * milliseconds, so the estimate would stay unknown on exactly the + * workload that needs it. + * 3. A reading that moves backwards in any of the three series -- which the + * clocks promise not to do, and which a test must still pin -- resets the + * baseline and reports unknown rather than subtracting into an underflow. + */ +int beam_competitors_from_samples(BeamAvailSample *prev, uint64_t sys, + uint64_t own, uint64_t wall, int allowed); + /** * The estimate the policy is given: free cores right now, at least one. * diff --git a/tests/native/test_avail.c b/tests/native/test_avail.c index 5596296..570c44e 100644 --- a/tests/native/test_avail.c +++ b/tests/native/test_avail.c @@ -138,10 +138,17 @@ static void test_free_cores_is_what_is_allowed_minus_what_is_running(void) { static void test_unknown_contention_falls_back_to_the_headroom_rule(void) { /* - * With no runnable count -- no /proc, as on Windows -- the estimate is - * everything allowed, and the policy's own halving then yields exactly the - * conservative constant that was wanted for that case. The fallback is not - * a special case anywhere; it falls out of the existing rule. + * With no runnable count the estimate is everything allowed, and clause 3 + * of the policy then caps it at the scaling ceiling -- so an unknown + * machine is planned at 8. The fallback is not a special case anywhere; it + * falls out of the existing rule. + * + * This comment used to credit "the policy's own halving" with that answer, + * which plan.c stopped doing when the ceiling form replaced the halving. + * It also used to say "as on Windows", which was the whole gap: Windows + * and macOS answered unknown on EVERY call, so they were planned at 8 + * whatever the machine was doing. They now difference two CPU-time + * readings instead, and reach this path only until a window accumulates. */ ASSERT_MSG(beam_available_from(16, -1) == 16, "unknown contention assumes none and lets the policy halve it"); @@ -150,6 +157,168 @@ static void test_unknown_contention_falls_back_to_the_headroom_rule(void) { PASS(); } +static void test_a_window_too_short_to_mean_anything_is_unknown(void) { + /* + * CPU time is accounted in ticks -- about 10 ms on both platforms that + * need this -- so a window of a few microseconds between two BEAM calls + * carries no information at all: the deltas are zero or one tick and the + * quotient is noise. Unknown, which the policy already handles as "assume + * no contention", is the honest answer and the existing safe fallback. + */ + ASSERT_MSG(beam_competitors_from_cpu_time(0u, 0u, 1000u, 16) == -1, + "a one-microsecond window cannot be measured"); + ASSERT_MSG(beam_competitors_from_cpu_time(100000000u, 0u, 5000000u, 16) == -1, + "five milliseconds is still below one accounting tick"); + PASS(); +} + +static void test_an_idle_machine_reports_only_ourselves(void) { + /* One second of wall, one core's worth of CPU, all of it ours. */ + ASSERT_MSG(beam_competitors_from_cpu_time(1000000000u, 1000000000u, + 1000000000u, 16) == 1, + "our own core does not compete with us"); + PASS(); +} + +static void test_competitors_are_counted_in_whole_cores(void) { + /* + * One second of wall during which the machine burned five core-seconds, + * one of them ours: four competitors, plus ourselves, is five runnable -- + * the same units beam_available_from takes. + */ + ASSERT_MSG(beam_competitors_from_cpu_time(5000000000u, 1000000000u, + 1000000000u, 16) == 5, + "four competing cores plus ourselves"); + ASSERT_MSG(beam_available_from(16, beam_competitors_from_cpu_time( + 5000000000u, 1000000000u, 1000000000u, 16)) == 12, + "and that leaves twelve of sixteen"); + PASS(); +} + +static void test_our_own_threads_are_not_competition(void) { + /* + * THE reason this function subtracts. A tick difference spanning BEAM's + * own eight-thread region reads the machine as eight cores busy, and + * without the subtraction the next call would plan narrow BECAUSE OF THE + * PREVIOUS ONE -- the feedback loop avail.h warns about for the load + * average, which "would serialise every call for a minute after BEAM's own + * previous call finished". Eight core-seconds, all ours, is an idle + * machine as far as the plan is concerned. + */ + ASSERT_MSG(beam_competitors_from_cpu_time(8000000000u, 8000000000u, + 1000000000u, 16) == 1, + "our own eight threads leave the machine ours"); + PASS(); +} + +static void test_a_saturated_machine_cannot_report_more_than_it_has(void) { + /* + * System CPU time covers every CPU, while `allowed` may be a subset -- + * the same approximation the runnable count already makes on Linux. The + * clamp keeps a hundred busy cores from producing a negative estimate, + * and the policy's own floor then yields one. + */ + const int nr = beam_competitors_from_cpu_time(100000000000u, 0u, + 1000000000u, 16); + ASSERT_MSG(nr == 17, "sixteen allowed, so at most sixteen competitors plus us"); + ASSERT_MSG(beam_available_from(16, nr) == 1, "a saturated machine leaves one"); + PASS(); +} + +static void test_a_fraction_of_a_core_rounds_towards_contention(void) { + /* + * Half a core of competition rounds up to one, not down to none. Planning + * too narrow costs about 2x and planning too wide cost up to 3814x, so + * the rounding goes the cheap way. + */ + ASSERT_MSG(beam_competitors_from_cpu_time(1500000000u, 0u, + 1000000000u, 16) == 3, + "one and a half competing cores counts as two, plus ourselves"); + PASS(); +} + +static void test_an_inconsistent_sample_is_not_negative_competition(void) { + /* + * The two clocks are read one after the other, not atomically, so our own + * CPU time can exceed the system total the sample before it. Reporting + * fewer than zero competitors would widen the plan on the strength of a + * sampling race. + */ + ASSERT_MSG(beam_competitors_from_cpu_time(1000000000u, 4000000000u, + 1000000000u, 16) == 1, + "more of our own time than the system's means no competition"); + PASS(); +} + +static void test_a_threads_first_reading_is_only_a_baseline(void) { + /* Nothing to difference against, so unknown -- and the reading is kept. */ + BeamAvailSample s = {0}; + ASSERT_MSG(beam_competitors_from_samples(&s, 5000000000u, 1000000000u, + 1000000000u, 16) == -1, + "the first reading cannot be interpreted"); + ASSERT_MSG(s.have == 1, "but it becomes the baseline"); + ASSERT_MSG(s.wall == 1000000000u, "stored as given"); + PASS(); +} + +static void test_a_short_window_leaves_the_baseline_to_age(void) { + /* + * The rule the whole mechanism depends on. A tight loop of short calls + * must not restart the window each time, or twenty milliseconds is never + * reached and the estimate stays unknown on exactly the workload that + * needs it. Three five-millisecond calls report unknown and leave the + * baseline where it was; the fourth, now twenty-five milliseconds from + * that baseline, answers. + */ + BeamAvailSample s = {0}; + (void)beam_competitors_from_samples(&s, 0u, 0u, 1000000000u, 16); + const uint64_t base = s.wall; + for (int i = 1; i <= 3; i++) { + const uint64_t at = 1000000000u + (uint64_t)i * 5000000u; + ASSERT_MSG(beam_competitors_from_samples(&s, 0u, 0u, at, 16) == -1, + "five milliseconds cannot be measured"); + ASSERT_MSG(s.wall == base, "and the baseline must not move"); + } + const int nr = beam_competitors_from_samples(&s, 50000000u, 0u, + 1025000000u, 16); + ASSERT_MSG(nr == 3, "two competing cores plus ourselves, once the window is real"); + ASSERT_MSG(s.wall == 1025000000u, "and now the baseline advances"); + PASS(); +} + +static void test_a_reading_that_moves_backwards_resets_rather_than_underflows(void) { + /* + * The clocks promise monotonicity; this pins what happens if one breaks + * it, because the subtraction is unsigned and the failure would not be a + * wrong answer but a vast one. + */ + BeamAvailSample s = {0}; + (void)beam_competitors_from_samples(&s, 9000000000u, 2000000000u, + 5000000000u, 16); + ASSERT_MSG(beam_competitors_from_samples(&s, 9000000000u, 2000000000u, + 4000000000u, 16) == -1, + "a wall clock that went backwards is unknown, not enormous"); + ASSERT_MSG(beam_competitors_from_samples(&s, 1000000000u, 2000000000u, + 9000000000u, 16) == -1, + "system time that went backwards is unknown too"); + PASS(); +} + +static void test_the_sampled_estimate_tracks_competitors_over_the_window(void) { + /* + * End to end through the stateful step: one second of wall in which the + * machine burned twelve core-seconds, four of them BEAM's own, leaves + * eight competitors and so eight of sixteen cores free. + */ + BeamAvailSample s = {0}; + (void)beam_competitors_from_samples(&s, 0u, 0u, 0u, 16); + const int nr = beam_competitors_from_samples(&s, 12000000000u, 4000000000u, + 1000000000u, 16); + ASSERT_MSG(nr == 9, "eight competing cores plus ourselves"); + ASSERT_MSG(beam_available_from(16, nr) == 8, "which leaves eight free"); + PASS(); +} + static void test_free_cores_is_never_below_one_or_above_what_is_allowed(void) { for (int allowed = -4; allowed <= 64; ++allowed) { for (int nr = -4; nr <= 80; ++nr) { @@ -288,6 +457,17 @@ int main(int argc, char **argv) { RUN_TEST(test_an_unreadable_cpu_max_is_reported_as_unknown); RUN_TEST(test_free_cores_is_what_is_allowed_minus_what_is_running); RUN_TEST(test_unknown_contention_falls_back_to_the_headroom_rule); + RUN_TEST(test_a_window_too_short_to_mean_anything_is_unknown); + RUN_TEST(test_an_idle_machine_reports_only_ourselves); + RUN_TEST(test_competitors_are_counted_in_whole_cores); + RUN_TEST(test_our_own_threads_are_not_competition); + RUN_TEST(test_a_saturated_machine_cannot_report_more_than_it_has); + RUN_TEST(test_a_fraction_of_a_core_rounds_towards_contention); + RUN_TEST(test_an_inconsistent_sample_is_not_negative_competition); + RUN_TEST(test_a_threads_first_reading_is_only_a_baseline); + RUN_TEST(test_a_short_window_leaves_the_baseline_to_age); + RUN_TEST(test_a_reading_that_moves_backwards_resets_rather_than_underflows); + RUN_TEST(test_the_sampled_estimate_tracks_competitors_over_the_window); RUN_TEST(test_free_cores_is_never_below_one_or_above_what_is_allowed); RUN_TEST(test_more_competitors_never_frees_more_cores); RUN_TEST(test_the_machine_reports_a_usable_estimate); diff --git a/verification/baseline/RERECORDED.md b/verification/baseline/RERECORDED.md index 05c6977..6670d22 100644 --- a/verification/baseline/RERECORDED.md +++ b/verification/baseline/RERECORDED.md @@ -3559,3 +3559,58 @@ new test uses `capfd` and not `capsys`, since the fallback writes to the file descriptor and never passes through `sys.stderr`; it fails when the fallback is removed. + +## Contention on macOS and Windows (2026-11-15) + +Two files moved, both inventory. No recorded NUMBER changed: `numeric`, +`math_values`, the saved-model sha256, thread independence and `exact` are all +byte-identical, which is the property that matters, since this change adds a +reading of the machine and must not touch an answer. + +### `c_cases.txt`: 439 -> 450 + +Eleven tests, all in `tests/native/test_avail.c`, for the estimate the +platforms without `/proc` now compute. Six exercise the pure arithmetic: + +- `test_a_window_too_short_to_mean_anything_is_unknown` +- `test_an_idle_machine_reports_only_ourselves` +- `test_competitors_are_counted_in_whole_cores` +- `test_our_own_threads_are_not_competition` +- `test_a_saturated_machine_cannot_report_more_than_it_has` +- `test_a_fraction_of_a_core_rounds_towards_contention` +- `test_an_inconsistent_sample_is_not_negative_competition` + +and four the sample-to-sample step, which carries the state: + +- `test_a_threads_first_reading_is_only_a_baseline` +- `test_a_short_window_leaves_the_baseline_to_age` +- `test_a_reading_that_moves_backwards_resets_rather_than_underflows` +- `test_the_sampled_estimate_tracks_competitors_over_the_window` + +Four of them were mutated against the real suite rather than merely written: +advancing the baseline on a short window, dropping the backwards guard, +dropping the subtraction of BEAM's own CPU time, and rounding a fraction of a +core down instead of up each fail exactly the test that names the property. +The third of those is the one that matters most -- without the subtraction a +window spanning BEAM's own eight-thread region reads as eight cores busy, so +each call would narrow the next because of the previous one, which is the +feedback loop `avail.h` rejects the load average for. + +(The mutation run also produced a false green worth recording: `cp -p` restored +the backup with its original mtime, `make` judged the object current, and the +"restored" run still reported the mutant's failure. The source was clean and +the binary was not. `touch` before the rebuild is what the file's own warning +about mtime granularity is for.) + +### `c_symbols.txt`: 261 -> 263 + +- `beam_competitors_from_cpu_time` +- `beam_competitors_from_samples` + +Both are declared in the internal header beside the existing +`beam_parse_nr_running` and `beam_available_from`, and are external for the +same reason those are: the tests link `libbeam.a` and call them directly. The +gate only NOTES a new internal symbol, so leaving these unrecorded would have +cost nothing today and 2 permanent notes thereafter -- which is how 30 symbols +drifted across four merges before they were recorded in #7. Previous versions +kept as `c_cases.availsample.txt` and `c_symbols.availsample.txt`. diff --git a/verification/baseline/c_cases.availsample.txt b/verification/baseline/c_cases.availsample.txt new file mode 100644 index 0000000..ca2ec55 --- /dev/null +++ b/verification/baseline/c_cases.availsample.txt @@ -0,0 +1,439 @@ +test_N_is_a_residue_and_other_codes_are_not +test_a_bad_shape_is_refused_at_construction +test_a_banded_logo_is_a_filtered_sum_over_the_support +test_a_bin_aligned_band_is_the_spectrum_over_those_bins +test_a_budget_too_small_refuses_and_says_why +test_a_budget_too_small_refuses_both_sweeps +test_a_ceiling_of_one_refuses_rather_than_undercounts +test_a_character_outside_the_alphabet_is_malformed_input +test_a_compacted_model_cannot_report_depth_occupancy +test_a_config_past_its_alphabets_ceiling_is_refused_by_name +test_a_conserved_position_is_dominated_but_not_one_hot +test_a_constrained_probability_matches_enumeration +test_a_constraint_over_the_wrong_alphabet_is_refused +test_a_count_probability_matches_enumeration +test_a_count_range_that_cannot_be_met_accepts_nothing +test_a_degenerate_range_yields_nothing +test_a_dense_model_finalizes_eagerly_to_the_same_bits +test_a_depth_bucket_with_no_rows_matches_the_definition +test_a_deterministic_dependency_carries_one_bit_as_the_prior_vanishes +test_a_dominant_rejected_path_cannot_collapse_the_answer +test_a_dropped_member_leaves_the_single_member_profile +test_a_finite_top_edge_still_measures_the_whole_grid +test_a_format_11_file_says_upgrade_beam +test_a_fractional_exponent_is_the_mellin_transform +test_a_functional_the_engine_cannot_sum_is_refused +test_a_grid_over_budget_is_refused +test_a_grid_too_coarse_is_refused_on_request +test_a_header_length_past_the_arrays_is_corrupt +test_a_k_way_product_is_the_mellin_transform_at_k +test_a_language_and_its_complement_partition_the_mass +test_a_layer_indexed_automaton_answers_a_disjunction +test_a_length_spelled_once_has_no_entropy_as_the_prior_vanishes +test_a_length_the_model_never_generates_is_refused +test_a_line_is_one_stretch_wherever_it_fits +test_a_loaded_model_keeps_no_count_directory +test_a_long_model_is_written_as_format_10 +test_a_matched_motif_cannot_be_un_matched +test_a_mismatched_automaton_is_refused_and_still_consumed +test_a_model_against_itself_has_coefficient_one +test_a_model_against_itself_is_one_over_its_hill_number +test_a_model_longer_than_the_old_ceiling_builds_and_scores +test_a_model_reports_the_alphabet_it_was_built_with +test_a_moment_sweep_is_cheaper_than_a_posterior +test_a_motif_conditional_matches_enumeration +test_a_motif_probability_matches_enumeration +test_a_motif_that_cannot_fit_is_refused +test_a_motif_the_corpus_never_spells_is_improbable_but_possible +test_a_narrow_budget_finalizes_to_the_same_bits +test_a_negative_tilt_does_not_overflow_the_sweeps +test_a_new_constraint_admits_every_residue +test_a_nonsense_pin_is_ignored_rather_than_obeyed +test_a_nucleotide_file_relabelled_as_amino_acid_is_rejected +test_a_nucleotide_model_builds_and_scores +test_a_nucleotide_model_past_the_byte_ceiling_builds_and_scores +test_a_nucleotide_model_round_trips_through_a_file +test_a_null_input_plans_one_thread +test_a_pair_joint_is_a_distribution_with_the_right_marginals +test_a_pin_above_the_cores_present_is_clamped +test_a_pinned_estimate_overrides_the_measurement +test_a_pinned_layer_is_estimated_cheaper_than_a_free_one +test_a_pinned_position_reproduces_a_clamp +test_a_pinned_probability_reproduces_log_clamp_probability +test_a_plan_does_not_reach_another_thread +test_a_plane_major_table_refuses_the_row_pointer +test_a_pooled_row_is_still_the_prior_of_its_counts +test_a_position_past_the_end_is_refused_by_name +test_a_product_past_the_state_ceiling_is_refused +test_a_query_length_past_the_old_ceiling_meets_the_budget +test_a_rare_but_certain_event_has_moments +test_a_refused_run_leaves_the_constraint_untouched +test_a_reserved_partition_never_grows +test_a_reserved_table_holds_what_a_grown_one_holds +test_a_residue_outside_the_alphabet_is_refused_by_name +test_a_run_applies_as_a_prefix_and_as_a_suffix +test_a_run_near_the_integer_ceiling_does_not_wrap +test_a_saturated_machine_plans_one_thread +test_a_set_conditions_as_the_mixture_of_its_members +test_a_set_probability_is_the_sum_over_its_members +test_a_short_model_is_still_written_as_format_9 +test_a_sink_below_the_grid_origin_still_reports_its_mass +test_a_source_at_the_length_ceiling_is_handled +test_a_source_the_model_cannot_score_is_refused +test_a_table_numbered_out_of_order_is_refused +test_a_top_k_search_cannot_split_its_way_past_the_budget +test_a_transposed_table_holds_the_prior_of_its_counts +test_a_transposed_table_reads_back_identically +test_a_universal_automaton_reproduces_the_mask_sweep +test_a_zero_exponent_drops_its_member +test_a_zero_forward_weight_leaves_no_nan_edge_weights +test_a_zeroed_config_still_means_amino_acid +test_accumulate_bins_and_conserves_mass +test_accumulator_max_ignores_an_earlier_threads_error +test_accumulator_max_is_not_the_surprisal +test_add_equals_counting_together +test_add_then_subtract_is_identity +test_all_ones_exponents_are_the_plain_inner_product +test_alphabet_names_are_distinct +test_ambiguity_beyond_N_is_still_rejected +test_an_amino_acid_ceiling_is_not_the_nucleotide_one +test_an_automaton_makes_a_constraint_not_free +test_an_empty_band_is_refused_not_answered +test_an_expected_count_is_the_column_of_marginals +test_an_explicit_cap_does_not_override_the_memory_cap +test_an_explicit_cap_is_honoured_exactly +test_an_explicit_cap_sizes_memory_for_itself +test_an_illegal_row_count_is_not_blamed_on_the_budget +test_an_impossible_constraint_has_probability_zero +test_an_impossible_neighbourhood_still_reports_its_odds +test_an_indicator_expectation_is_the_marginal +test_an_omp_env_setting_is_not_overridden +test_an_omp_env_setting_sizes_memory_for_itself +test_an_over_length_sequence_scores_minus_infinity +test_an_over_length_source_is_probability_zero_up_to_the_buffer +test_an_unconstrained_query_reproduces_the_marginals +test_an_unfinalized_member_is_refused +test_an_unknown_alphabet_in_a_file_is_refused +test_an_unknown_core_count_still_covers_the_plan +test_an_unreadable_cpu_max_is_reported_as_unknown +test_an_unreadable_loadavg_is_reported_as_unknown +test_and_or_and_not_are_the_boolean_combinations +test_attaching_an_automaton_intersects +test_auto_plans_the_free_cores_up_to_the_scaling_ceiling +test_availability_above_the_core_count_is_clamped +test_backward_context_from_state_matches_the_sequence +test_backward_context_packing +test_backward_wide_context_matches_the_sequence +test_bad_exponents_are_refused +test_batch_matches_scalar +test_beam_threads_is_stable_inside_a_plan_scope +test_beyond_the_ceiling_more_free_cores_buy_nothing +test_beyond_the_reduction_range +test_bits_do_not_depend_on_the_call_size +test_both_accessors_agree +test_both_crc_paths_match_the_polynomial +test_both_formats_load +test_both_union_entry_points_enforce_the_same_ceiling +test_budget_concurrency_follows_the_active_plan +test_budget_refusal_does_not_poison_the_model +test_builders_commute +test_builders_intersect_rather_than_replace +test_c_anchor_penalises_a_bad_terminus +test_case_folded_input_does_not_multiply_the_neighbours +test_clamped_marginals_survive_a_wide_dynamic_range +test_combining_incompatible_automata_is_refused +test_compact_keeps_scores +test_company_threads_and_budget_do_not_change_the_bits +test_conditioning_on_the_impossible_is_refused +test_construction_refuses_an_unknown_alphabet +test_contradictory_builders_leave_an_empty_position +test_corrupt_file_is_rejected +test_crc32c_table_matches_the_polynomial +test_cumulants_follow_from_moments +test_de_morgan_holds_over_the_whole_language +test_defaults_are_the_measured_ones +test_defaults_refuse_an_unknown_alphabet +test_defaults_sit_inside_their_own_ceilings +test_degenerate_input_is_defined +test_deletion_neighbours_collapse_runs +test_dense_best_completion_never_exceeds_the_total +test_dense_matches_the_definition +test_dense_sweeps_each_compute_their_own_semiring +test_depth_occupancy_sums_to_the_nonzero_cells +test_descriptors_are_internally_consistent +test_double_negation_is_the_identity +test_dropping_an_aliased_cache_frees_each_list_once +test_each_neighbour_is_attributed_to_its_leftmost_gap +test_effective_threads_reports_the_cap_not_the_plan +test_either_storage_layout_is_a_valid_configuration +test_entropy_matches_enumeration +test_entropy_reproduces_the_models_own_surprisal_moments +test_every_byte_maps_exactly_as_specified +test_every_cell_is_still_the_prior_of_its_counts +test_every_config_refusal_names_its_own_rule +test_every_descriptor_fits_the_length_constants +test_every_entry_point_refuses_the_impossible +test_every_layer_of_a_real_model_aliases +test_every_legal_pair_accepts_exactly_up_to_its_own_bound +test_every_lifecycle_operation_survives_frozen_counts +test_every_nucleotide_order_finalizes_and_scores +test_every_order_is_its_single_order_value_bitwise +test_every_position_agrees_on_the_band_mass +test_every_route_matches_the_reference_bit_for_bit +test_every_tier_boundary +test_exp_bits_do_not_depend_on_the_call_size +test_exp_is_exact_at_zero_and_flushes_below_the_normals +test_exp_within_two_ulp +test_explain_reconstructs_the_score +test_failures_carry_a_message +test_finalize_picks_the_representation_per_table +test_finalize_rolls_back_on_allocation_failure +test_finalize_transposes_only_the_backward_top_table +test_fingerprint_partition_matches_sequence_enumeration +test_fingerprint_refusals_and_root_only +test_forward_context_packing +test_fractional_counts_round_trip +test_free_cores_is_never_below_one_or_above_what_is_allowed +test_free_cores_is_what_is_allowed_minus_what_is_running +test_generation_emits_only_this_alphabet +test_grouped_accumulation_matches_independent_rows +test_growth_overshoots_and_reallocates +test_hill_matches_mellin +test_hill_numbers +test_identical_members_have_one_members_support +test_independent_positions_share_no_information_as_the_prior_vanishes +test_index_and_character_round_trip +test_information_refuses_what_it_cannot_answer +test_inserting_then_deleting_returns_the_source +test_insertion_neighbours_are_the_distinct_strings +test_integer_counts_round_trip +test_interning_changes_no_row +test_invalid_inputs_are_reported +test_it_refuses_only_where_one_order_would_not_fit +test_large_orders_stay_finite_and_in_bounds +test_layer_max_is_unchanged_after_sealing +test_layer_weights_reproduce_the_scorer +test_lazy_and_eager_finalize_agree +test_leaving_a_default_scope_restores_the_previous_count +test_leaving_a_plan_scope_restores_the_previous_value +test_legacy_format_files_are_refused_by_name +test_legal_rows_with_real_memory_demand_get_the_budget_answer +test_length_derivatives_mix_to_the_pooled_moments +test_length_marginals_decompose +test_length_prior_normalises +test_lifting_masks_preserves_the_language +test_line_agrees_with_the_general_form +test_line_bits_depend_on_the_line_alone +test_line_plans_cover_the_line_within_the_budget +test_live_cache_is_dropped_by_every_mutation +test_live_cache_respects_the_memory_bound +test_live_cache_returns_identical_answers +test_live_frontier_invariants_hold_at_every_order +test_local_distance_different_maximum_lengths +test_local_distance_disjoint_length_support +test_local_distance_matches_enumeration_all_widths +test_local_distance_refusals +test_local_distance_tie_order +test_local_distance_topk_symmetry_identity_and_threads +test_log_mellin_endpoints +test_log_z_splits_across_its_layers +test_lower_case_scores_as_upper_case +test_lower_case_trains_as_upper_case +test_many_rare_paths_still_mix_into_an_entropy +test_marginal_rows_sum_to_one_under_a_constraint +test_masks_and_an_automaton_compose +test_matches_exhaustive_sequence_law +test_matches_the_enumerated_sum +test_max_length_is_free_up_to_the_ceiling +test_members_of_different_shapes_are_refused +test_members_trained_apart_match_the_enumeration +test_model_algebra_refuses_across_alphabets +test_moments_match_enumeration_under_a_constraint +test_moments_match_the_reference_pmf +test_moments_of_an_impossible_constraint_do_not_exist +test_moments_respect_an_automaton +test_more_competitors_never_frees_more_cores +test_more_free_cores_never_plans_fewer_threads +test_multiplicity_accounts_for_every_edit_exactly_once +test_narrow_window_fails_loudly +test_near_multiples_of_half_pi +test_neighbourhoods_at_the_short_extreme +test_no_cgroup_limit_is_reported_as_unknown +test_no_character_encodes_to_the_sentinel +test_no_dense_sweep_loses_a_state +test_no_more_threads_than_work_items +test_no_sparse_sweep_loses_a_state +test_no_threads_plans_as_one_thread +test_nucleotides_fold_case_and_amino_acids_do_not +test_observe_counts_and_rejects +test_one_band_over_everything_is_the_exact_marginal +test_one_band_over_the_grid_is_exact +test_one_block_can_be_released_before_the_rest +test_one_member_sums_to_one +test_only_an_unreadable_sequence_scores_minus_infinity +test_order_four_rows_are_normalized_and_log_consistent +test_packed_matches_scalar_and_refuses_bad_offsets +test_partition_normalises +test_per_alphabet_defaults_come_from_the_descriptor +test_pgf_inverts_to_the_reference_pmf +test_plan_scopes_nest +test_posterior_counts_round_trip +test_posterior_of_a_model_with_itself_keeps_its_scores +test_posterior_shrinks_towards_the_prior +test_prior_strengths_past_the_order_are_ignored +test_provenance_is_recorded_and_aggregated +test_random_angles_within_two_ulp +test_real_axis_and_conjugates +test_real_plans_cover_the_orders_within_the_budget +test_refining_the_grid_does_not_move_a_band +test_rejects_bad_arguments +test_rejects_unusable_sequences +test_resident_memory_stays_near_the_model_size +test_residue_indices_are_dense +test_results_do_not_depend_on_the_thread_count +test_rows_are_identified_the_cheaper_way +test_rows_that_differ_are_kept_apart +test_sampling +test_sampling_is_reproducible +test_save_load_round_trip +test_saving_a_frozen_model_writes_the_same_bytes +test_scores_are_bit_identical_after_sealing +test_scores_are_bit_identical_with_planes +test_scores_are_finite_and_ordered +test_scores_are_those_of_the_dense_model +test_scores_are_those_of_the_dense_model +test_sealing_changes_no_row +test_single_anchor_weights_are_accepted +test_slot0_counts_the_rows_before_it +test_sparse_best_completion_never_exceeds_the_total +test_sparse_combined_max_is_over_the_marked_rows +test_sparse_growth_failure_is_atomic +test_sparse_matches_dense_bit_for_bit +test_sparse_order_2_matches_the_definition +test_sparse_order_4_matches_the_definition +test_sparse_rows_boundaries_and_erase +test_sparse_sweeps_each_compute_their_own_semiring +test_spectra_by_length_equal_one_at_a_time +test_spectrum_pair_equals_two_spectra +test_spectrum_totals_and_identity +test_subtract_is_the_additive_inverse +test_subtract_refuses_to_empty_the_model +test_subtract_restores_the_sequence_counters +test_summed_position_entropies_never_undercut_the_sequence +test_support_is_all_of_sequence_space +test_surprisal_mean_is_the_entropy +test_surprisal_moments_by_enumeration +test_surprisal_range_by_enumeration +test_the_accumulators_refuse_and_say_it_was_the_budget +test_the_alias_is_taken_exactly_when_the_lists_agree +test_the_allocator_policy_is_applied +test_the_alphabets_overlap_with_different_indices +test_the_amino_acid_defaults_are_unchanged +test_the_analytics_match_enumeration_for_nucleotides +test_the_automaton_allocation_respects_the_memory_budget +test_the_band_masses_account_for_the_whole_spectrum +test_the_banding_matches_brute_force_on_a_tiny_model +test_the_bands_recombine_into_the_unconditional_marginal +test_the_bhattacharyya_coefficient_matches_the_enumeration +test_the_byte_count_drops_by_the_aliased_lists +test_the_calling_contract +test_the_calling_contract +test_the_cgroup_quota_is_read_as_whole_cores +test_the_compaction_report_matches_what_the_table_became +test_the_company_an_order_keeps_does_not_change_it +test_the_cost_estimate_precedes_the_sweep +test_the_count_automaton_is_exactly_a_tally +test_the_count_automaton_uses_no_more_states_than_it_needs +test_the_cursor_matches_the_predicate_when_direct +test_the_cursor_matches_the_predicate_when_sealed +test_the_default_prior_leaves_no_position_certain_and_no_dependency_exact +test_the_default_scope_honours_an_explicit_cap +test_the_default_scope_plans_from_the_pinned_availability +test_the_default_scope_stands_aside_for_omp_num_threads +test_the_default_scope_stops_at_the_ceiling +test_the_dictionary_holds_each_value_once +test_the_dissimilarity_profile_sums_to_a_nonnegative_distance +test_the_doubling_paths_refuse_too +test_the_driver_and_the_sweep_agree_on_the_size +test_the_estimate_does_not_exceed_the_cores_present +test_the_estimate_grows_with_the_automaton +test_the_estimate_matches_what_the_sweep_allocates +test_the_fine_grid_loses_mass_off_its_top +test_the_gram_matrix_matches_the_pairs_one_at_a_time +test_the_guard_divides_exactly_as_the_operator_does +test_the_guard_notices_a_value_that_vanishes +test_the_information_cost_reports_the_whole_job +test_the_information_matrix_matches_enumeration +test_the_layout_count_covers_the_team_with_no_plan_active +test_the_layout_count_holds_still_as_the_machine_fills +test_the_layout_count_ignores_how_busy_the_machine_is +test_the_layout_count_never_undercuts_the_plan +test_the_machine_reports_a_usable_estimate +test_the_matrix_agrees_with_the_pair_joints +test_the_matrix_obeys_the_identities_information_must +test_the_matrix_under_a_constraint_matches_enumeration +test_the_memory_cap_is_never_exceeded +test_the_mixture_sweep_loses_nothing_either +test_the_moment_sweep_agrees_about_the_partition +test_the_motif_automaton_is_exactly_substring_search +test_the_odds_are_the_scores_they_claim_to_be +test_the_order_bound_is_the_alphabet_s_own +test_the_order_ceiling_keeps_context_counts_in_range +test_the_partition_normalises_above_order_four +test_the_partition_normalises_over_nucleotides +test_the_per_length_terms_sum_to_the_total +test_the_per_position_terms_sum_to_the_total +test_the_plan_never_drops_below_one +test_the_policy_applies_without_being_named +test_the_positional_profile_does_not_depend_on_member_order +test_the_positional_split_leaves_the_total_bit_identical +test_the_positional_total_agrees_with_the_by_length_vector +test_the_posterior_ranks_the_insertions_the_corpus_prefers +test_the_posterior_survives_a_neighbourhood_that_underflows +test_the_power_gram_matches_the_pairs_one_at_a_time +test_the_probability_of_everything_is_exactly_one +test_the_recovered_mean_surprisal_is_unbiased +test_the_reference_alphabet_sets_the_ceiling +test_the_registry_refuses_what_it_does_not_know +test_the_registry_resolves_ids_and_names +test_the_reported_figure_tracks_the_work +test_the_reported_working_set_covers_every_allocation +test_the_reported_working_set_is_not_a_meaningless_ceiling +test_the_runnable_count_is_read_from_the_fourth_field +test_the_same_input_always_plans_the_same_count +test_the_selfcheck_passes_on_an_aliased_cache +test_the_shipped_budget_answers_and_every_block_is_whole +test_the_stack_identities_hold +test_the_state_ceiling_admits_what_it_says_it_does +test_the_state_is_exactly_one_context_wide +test_the_streaming_log_sum_exp_matches_a_two_pass_reference +test_the_sweep_and_the_accumulators_share_one_budget +test_the_table_accessors_refuse_what_they_cannot_read +test_the_top_k_merge_buffer_is_refused_before_it_is_taken +test_the_transpose_actually_moves_the_cells +test_the_two_alphabets_do_not_interfere +test_the_unconditional_logo_is_a_direct_sum_over_the_support +test_the_union_ceiling_binds_only_the_union +test_the_union_never_exceeds_the_pooled_model +test_the_union_of_any_members_is_the_whole_space +test_the_whole_engine_on_a_nucleotide_model +test_threads_and_budget_do_not_change_an_order +test_threads_setting_round_trips +test_top_k_by_enumeration +test_top_sequence_attains_the_minimum_surprisal +test_truncation_discards_the_far_tail +test_two_bands_recombine_into_the_exact_logo +test_union_obeys_inclusion_exclusion +test_unknown_contention_falls_back_to_the_headroom_rule +test_validation_refuses_an_unknown_alphabet +test_variance_survives_a_large_constant_offset +test_variance_survives_an_offset_that_squares_to_infinity +test_what_a_malformed_band_gets +test_what_a_malformed_request_gets +test_what_cannot_be_interned_is_left_alone +test_what_cannot_be_packed_is_left_alone +test_what_cannot_be_sealed_is_left_alone +test_what_cannot_be_transposed_is_left_alone +test_what_reserve_refuses_and_ignores +test_with_no_plan_active_nothing_changes +test_zero_is_exact diff --git a/verification/baseline/c_cases.txt b/verification/baseline/c_cases.txt index ca2ec55..7c122bd 100644 --- a/verification/baseline/c_cases.txt +++ b/verification/baseline/c_cases.txt @@ -21,6 +21,7 @@ test_a_dominant_rejected_path_cannot_collapse_the_answer test_a_dropped_member_leaves_the_single_member_profile test_a_finite_top_edge_still_measures_the_whole_grid test_a_format_11_file_says_upgrade_beam +test_a_fraction_of_a_core_rounds_towards_contention test_a_fractional_exponent_is_the_mellin_transform test_a_functional_the_engine_cannot_sum_is_refused test_a_grid_over_budget_is_refused @@ -67,24 +68,29 @@ test_a_position_past_the_end_is_refused_by_name test_a_product_past_the_state_ceiling_is_refused test_a_query_length_past_the_old_ceiling_meets_the_budget test_a_rare_but_certain_event_has_moments +test_a_reading_that_moves_backwards_resets_rather_than_underflows test_a_refused_run_leaves_the_constraint_untouched test_a_reserved_partition_never_grows test_a_reserved_table_holds_what_a_grown_one_holds test_a_residue_outside_the_alphabet_is_refused_by_name test_a_run_applies_as_a_prefix_and_as_a_suffix test_a_run_near_the_integer_ceiling_does_not_wrap +test_a_saturated_machine_cannot_report_more_than_it_has test_a_saturated_machine_plans_one_thread test_a_set_conditions_as_the_mixture_of_its_members test_a_set_probability_is_the_sum_over_its_members test_a_short_model_is_still_written_as_format_9 +test_a_short_window_leaves_the_baseline_to_age test_a_sink_below_the_grid_origin_still_reports_its_mass test_a_source_at_the_length_ceiling_is_handled test_a_source_the_model_cannot_score_is_refused test_a_table_numbered_out_of_order_is_refused +test_a_threads_first_reading_is_only_a_baseline test_a_top_k_search_cannot_split_its_way_past_the_budget test_a_transposed_table_holds_the_prior_of_its_counts test_a_transposed_table_reads_back_identically test_a_universal_automaton_reproduces_the_mask_sweep +test_a_window_too_short_to_mean_anything_is_unknown test_a_zero_exponent_drops_its_member test_a_zero_forward_weight_leaves_no_nan_edge_weights test_a_zeroed_config_still_means_amino_acid @@ -103,9 +109,11 @@ test_an_expected_count_is_the_column_of_marginals test_an_explicit_cap_does_not_override_the_memory_cap test_an_explicit_cap_is_honoured_exactly test_an_explicit_cap_sizes_memory_for_itself +test_an_idle_machine_reports_only_ourselves test_an_illegal_row_count_is_not_blamed_on_the_budget test_an_impossible_constraint_has_probability_zero test_an_impossible_neighbourhood_still_reports_its_odds +test_an_inconsistent_sample_is_not_negative_competition test_an_indicator_expectation_is_the_marginal test_an_omp_env_setting_is_not_overridden test_an_omp_env_setting_sizes_memory_for_itself @@ -144,6 +152,7 @@ test_clamped_marginals_survive_a_wide_dynamic_range test_combining_incompatible_automata_is_refused test_compact_keeps_scores test_company_threads_and_budget_do_not_change_the_bits +test_competitors_are_counted_in_whole_cores test_conditioning_on_the_impossible_is_refused test_construction_refuses_an_unknown_alphabet test_contradictory_builders_leave_an_empty_position @@ -272,6 +281,7 @@ test_one_block_can_be_released_before_the_rest test_one_member_sums_to_one test_only_an_unreadable_sequence_scores_minus_infinity test_order_four_rows_are_normalized_and_log_consistent +test_our_own_threads_are_not_competition test_packed_matches_scalar_and_refuses_bad_offsets test_partition_normalises test_per_alphabet_defaults_come_from_the_descriptor @@ -401,6 +411,7 @@ test_the_reported_working_set_covers_every_allocation test_the_reported_working_set_is_not_a_meaningless_ceiling test_the_runnable_count_is_read_from_the_fourth_field test_the_same_input_always_plans_the_same_count +test_the_sampled_estimate_tracks_competitors_over_the_window test_the_selfcheck_passes_on_an_aliased_cache test_the_shipped_budget_answers_and_every_block_is_whole test_the_stack_identities_hold diff --git a/verification/baseline/c_symbols.availsample.txt b/verification/baseline/c_symbols.availsample.txt new file mode 100644 index 0000000..74feea1 --- /dev/null +++ b/verification/baseline/c_symbols.availsample.txt @@ -0,0 +1,261 @@ +beam_account_release +beam_account_room +beam_account_take +beam_allowed_cores +beam_alphabet_by_name +beam_alphabet_info +beam_applied_mmap_threshold +beam_available_cores +beam_available_from +beam_best_completions +beam_beta_contour_sums +beam_beta_contour_tail +beam_betas +beam_betas_tilted +beam_binomials +beam_clear_error +beam_cohort_private_sums +beam_compute_log_z +beam_config_invalid_reason +beam_config_valid +beam_cores_present +beam_count_block_free +beam_crc32c +beam_crc32c_software +beam_crc32c_update +beam_encode +beam_encode_n +beam_error_message +beam_fb_advance +beam_fb_alloc +beam_fb_betas +beam_fb_free +beam_fingerprint_free +beam_fingerprint_search +beam_free_counts +beam_free_tables +beam_frozen_cursor_init +beam_frozen_cursor_next +beam_frozen_destroy +beam_frozen_freeze +beam_frozen_memory_bytes +beam_frozen_row +beam_frozen_row_id +beam_frozen_thaw +beam_insights_logo_cost +beam_insights_logo_set_free +beam_insights_logos +beam_last_error +beam_layer_max +beam_layer_weights +beam_layer_weights_live +beam_layout_threads +beam_line_plan +beam_line_work_free +beam_line_work_new +beam_live_acquire +beam_live_cache_bytes +beam_live_cache_drop +beam_live_cache_entries +beam_live_cache_lookup +beam_live_cache_publish +beam_live_free +beam_live_lengths_desc +beam_live_release +beam_live_selfcheck +beam_local_distance +beam_local_distance_free +beam_log_gamma_ratio +beam_log_gamma_ratio_shift +beam_logo_request_default +beam_mellin_terms +beam_memory_init +beam_mix +beam_mixture_check_shape +beam_mixture_gram +beam_mixture_log_inner_product +beam_mixture_log_inner_product_by_length +beam_mixture_log_power_product +beam_mixture_log_power_product_by_length +beam_mixture_log_power_product_by_position +beam_mixture_power_gram +beam_mixture_power_pairs +beam_mixture_support_union +beam_mixture_support_union_by_length +beam_model_accumulator_max +beam_model_add +beam_model_alphabet +beam_model_compact +beam_model_config +beam_model_config_default +beam_model_config_default_for +beam_model_depth_occupancy +beam_model_destroy +beam_model_drop_caches +beam_model_effective_threads +beam_model_entropy +beam_model_expected_counts +beam_model_expected_counts_free +beam_model_explain +beam_model_file_provenance_span +beam_model_finalize +beam_model_freeze_counts +beam_model_frozen_bytes +beam_model_frozen_table +beam_model_hill_number +beam_model_is_finalized +beam_model_length_derivatives +beam_model_length_marginals +beam_model_live_cache_bytes +beam_model_load +beam_model_log_length_prior +beam_model_log_mellin +beam_model_log_mellin_complex +beam_model_log_mellin_line +beam_model_log_mellin_many +beam_model_log_partition +beam_model_log_partition_by_layer +beam_model_log_pseq +beam_model_log_pseq_batch +beam_model_log_pseq_packed +beam_model_memory_budget +beam_model_memory_bytes +beam_model_n_cells +beam_model_n_nonzero_cells +beam_model_n_observed +beam_model_n_residues +beam_model_n_skipped +beam_model_n_states +beam_model_new +beam_model_observe +beam_model_path_count +beam_model_path_count_total +beam_model_positional_banded +beam_model_positional_marginals +beam_model_posterior +beam_model_provenance +beam_model_radix +beam_model_release_frozen +beam_model_sample +beam_model_save +beam_model_set_memory_budget +beam_model_set_prior +beam_model_set_threads +beam_model_spectra_by_length +beam_model_spectrum +beam_model_spectrum_pair +beam_model_subtract +beam_model_surprisal_by_depth +beam_model_surprisal_cumulants +beam_model_surprisal_moments +beam_model_surprisal_range +beam_model_surprisal_tilt +beam_model_temper +beam_model_thaw_count_table +beam_model_thaw_counts +beam_model_threads +beam_model_top_k +beam_model_topology +beam_model_total_weight +beam_model_transition_rows +beam_parse_cpu_max +beam_parse_nr_running +beam_plan_enter +beam_plan_enter_default +beam_plan_inputs +beam_plan_layout_threads +beam_plan_leave +beam_plan_max_width +beam_plan_threads +beam_positional_banded_cost +beam_publicness_accumulate +beam_publicness_accumulate_grouped +beam_publicness_moments +beam_publicness_pgf +beam_query_advance +beam_query_advance_product +beam_query_all_free +beam_query_automaton_accepts +beam_query_automaton_and +beam_query_automaton_clone +beam_query_automaton_count +beam_query_automaton_free +beam_query_automaton_from_masks +beam_query_automaton_layers +beam_query_automaton_motif +beam_query_automaton_not +beam_query_automaton_or +beam_query_automaton_states +beam_query_automaton_step +beam_query_betas +beam_query_betas_product +beam_query_check_site +beam_query_constrain_automaton +beam_query_constrain_exclude +beam_query_constrain_position +beam_query_constrain_run +beam_query_constrain_set +beam_query_constraint_automaton +beam_query_constraint_clone +beam_query_constraint_free +beam_query_constraint_is_free +beam_query_constraint_length +beam_query_constraint_mask +beam_query_constraint_new +beam_query_constraint_residues +beam_query_deletions +beam_query_empty_position +beam_query_entropy +beam_query_estimate +beam_query_expect +beam_query_information_cost +beam_query_information_matrix +beam_query_insertions +beam_query_log_probability +beam_query_marginals +beam_query_neighbourhood_free +beam_query_pair_joint +beam_query_position_entropy +beam_query_resolve +beam_query_work_alloc +beam_query_work_free +beam_raw_log_weight +beam_real_plan +beam_real_work_free +beam_real_work_new +beam_rescale_flush_reset +beam_rescale_flushes +beam_rescale_note_flush +beam_rng_seed +beam_set_error +beam_sparse_compact_finalized +beam_sparse_count_positive_cells +beam_sparse_destroy +beam_sparse_erase +beam_sparse_init +beam_sparse_intern_rows +beam_sparse_memory_bytes +beam_sparse_n_rows +beam_sparse_occupied_rows +beam_sparse_pack_cells +beam_sparse_plane_transpose +beam_sparse_reserve +beam_sparse_row +beam_sparse_row_mut +beam_sparse_seal_rank +beam_sparse_shrink +beam_spectrum_spacing +beam_state_weights +beam_surprisal_range_at +beam_tail_weights +beam_threads +beam_ztp_bin_expect +beam_ztp_bin_reduce +beam_ztp_count_spectrum +jet_sweep +jet_sweep_cfan +jet_sweep_fan +jet_sweep_line +jet_sweep_real +publicness_check_groups +publicness_check_probability diff --git a/verification/baseline/c_symbols.txt b/verification/baseline/c_symbols.txt index 74feea1..8075461 100644 --- a/verification/baseline/c_symbols.txt +++ b/verification/baseline/c_symbols.txt @@ -15,6 +15,8 @@ beam_betas_tilted beam_binomials beam_clear_error beam_cohort_private_sums +beam_competitors_from_cpu_time +beam_competitors_from_samples beam_compute_log_z beam_config_invalid_reason beam_config_valid