Explore seeds across worker processes - #15
Merged
Merged
Conversation
explore() takes jobs=N and pytest takes --simloop-jobs=N: seed batches fan out over a spawn-context process pool, and a batch frontier makes parallelism invisible in the answer — a failure only counts once every batch that could hold an earlier one has finished, so the reported seed is exactly the one a sequential sweep would have stopped at. Workers send back positions and exception summaries, never traces; the parent re-runs the failing seed (and the passing one below it) to build the report, and a seed that behaves differently across processes raises a determinism error instead of being papered over. Under pytest the worker imports the test module by path and unwraps the decorated test; tests that need fixtures refuse parallel mode by name. A benchmark script measures the fan-out.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
explore(fn, seeds, jobs=N)andpytest --simloop-jobs=Nfan seed batches out over a spawn-contextProcessPoolExecutor. The invariant is that parallelism never changes the answer: a batch frontier only accepts a failure once every batch that could contain an earlier seed has finished, so the reported seed is exactly the one a sequential sweep stops at — proven by unit tests over crafted completion orders plus an end-to-end agreement test.Workers return positions and exception summaries only. The parent re-runs the failing seed (and the passing seed just below it, for the trace diff) to build the full report — which doubles as a cross-process replay check: a seed that does something different in the parent raises
ParallelDeterminismErrorloudly.Under pytest, workers import the test module by file path and unwrap the
sim_testdecoration; tests taking fixtures refuse parallel mode with a clear message rather than silently degrading. Workloads must be picklable forjobs>1, checked up front with an error that names the constraint.Measured here (M4, 10 cores): the jobqueue 300-seed chaos campaign drops 5.5 s → 1.9 s under
--simloop-jobs=8; a synthetic sweep does ~226 → ~598 seeds/s at 10 jobs.benchmarks/parallel.pyreproduces the measurement; the numbers doc updates separately with the campaign results.Sequential path is a pure refactor — determinism suite untouched and passing. 262 fast + 10 slow, mypy strict clean.