An AI-assisted CPU graph-compiler prototype built under Jacob Garcia's direction on 17 October 2026. This is a new, narrow runnable project, not a claim of prior production ML-compiler experience.
Version 1 is preserved in v1/, including its original sources, tests and measured JSON. New measurements below use version 2. CSE is explicitly opt-in with --cse; old invocations remain valid.
Requirements: Python 3.10+, NumPy, GCC on Linux. No GPU, account, paid service or model download.
python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install -r requirements.txt
python3 -m unittest -v
python3 benchmark_upgrade.py --rounds 21 --output my-upgrade.json
python3 compiler.py two_branch.json --mode unfused --cse --out build-cse
python3 benchmark.py --model polynomial.json --rounds 31 --json my-polynomial.json
python3 benchmark.py --model model.json --rounds 31 --json my-activation.json
python3 compiler.py polynomial.json --mode fused --out build-polynomial
python3 compiler.py polynomial.json --mode unfused --out build-polynomialThe compiler writes C and compiles a shared library; the benchmark loads it through ctypes. Compiled libraries are intentionally not shipped: compile on your own Linux machine.
- Validate identifiers, allowed operations, arity, references, DAG acyclicity, scalar/vector types and vector output. Even dead code is validated before it is removed.
- Topologically order the graph. Constants are scalar float32; input tensors are contiguous one-dimensional float32 vectors of the same length.
- Constant-fold scalar expressions with float32 rounding.
- Mark dependencies backward from the output and remove dead nodes.
- Lower the same optimized graph either to one fused loop with scalar SSA-style locals or to nine separate loops and eight intermediate arrays.
- Compile both variants using the same GCC flags, without fast-math or FMA contraction. GCC supplies machine-code generation and vectorization.
Supported operations: input, const, add, mul, relu. This is not MLIR, XLA, TVM, an ONNX importer, automatic differentiation, a general tensor compiler or a GPU compiler. It does not implement reductions, matrix multiplication, dynamic shapes or cost-based fusion.
polynomial.json: a two-input pointwise residual polynomial. Let s = 0.8x - 0.3y + 0.375; output = 0.1*(ss)(s*s) + x + y. The explicit IR contains nine live vector operations. A scalar bias add is folded, and one unused vector operation plus two now-unneeded constants are removed. No dummy add-zero operations were added to inflate the baseline.
model.json: a ReLU residual graph. Let h = relu(0.8x - 0.3y + 0.375); output = relu(0.1hh + x). This workload exposes a failure of unconditional fusion.
CPU: Intel(R) Xeon(R) Processor @ 2.60GHz. gcc (Ubuntu 11.4.0-1ubuntu1~22.04.3) 11.4.0. Python 3.10.12, NumPy 2.2.6.
Main run:
| Elements | Unfused median ms | Fused median ms | Unfused / fused |
|---|---|---|---|
| 1,024 | 0.002154 | 0.001163 | 1.851x |
| 65,536 | 0.133789 | 0.029303 | 4.566x |
| 1,048,576 | 7.307387 | 0.493305 | 14.813x |
A second full run produced speedups of 1.825x, 4.513x, 14.497x in the same size order. Raw per-round samples are included in both JSON files.
Counterexample, ReLU graph:
| Elements | Unfused median ms | Fused median ms | Unfused / fused |
|---|---|---|---|
| 1,024 | 0.002171 | 0.002186 | 0.993x |
| 65,536 | 0.128161 | 0.355508 | 0.361x |
| 1,048,576 | 5.719159 | 5.899921 | 0.969x |
The fused ReLU loop did not vectorize because of control flow, as reported by GCC. Separate passes can be faster despite using more memory. The polynomial loop did vectorize (16-byte and 8-byte vectors). Fusion alone is not a universal optimization; a real compiler needs a profitability model and backend-aware partitioning.
One Python process, one CPU thread, deterministic normal-distributed float32 inputs, preallocated inputs/outputs/workspace, five warmups, 31 samples. Each sample repeats enough calls to target roughly 12 ms using an initial fused-call estimate, capped at 5000 repetitions. Order alternates each round. Reported times are the median per-call time; Python loop and ctypes call overhead are included equally. No allocation or compilation is timed. Inputs are reused, so this is a warm-input microbenchmark, not cold-cache end-to-end model serving.
Measurements ran on a shared Linux CPU host, without CPU pinning or isolated scheduling. They may vary on another run or machine. Speedups apply only to these small graphs and this baseline. At 1,048,576 elements the unfused workspace is 32 MiB; fused extra workspace is zero (inputs and outputs still consume memory).
Eleven unittest cases passed, covering malformed graphs, duplicate/unsafe identifiers, cycles, unknown dependencies, arity, non-finite constants, pass counts and both graph workloads. Both lowerings were checked at vector lengths 0, 1, 7, 31, 1024 and 65537. Benchmarked outputs exactly matched between the two C versions, and all tested NumPy-reference errors were zero.
Floating-point support is limited to finite, moderate-range values whose intermediates remain finite. NaN/Inf propagation and signed-zero fidelity are not a verified contract. The raw C ABI trusts its caller: all vector lengths and buffer sizes must match, buffers must be contiguous, and input/output/workspace arrays must not alias. The benchmark supplies valid buffers; the generated C entrypoint does not bounds-check arbitrary pointers. Do not expose it directly to untrusted callers.
This artifact demonstrates a small typed graph IR, deterministic compiler passes, C lowering, a runnable correctness suite, and measurement of both a successful optimization and an important regression. It does not establish broad infrastructure expertise, GPU experience, production reliability, model quality or experience shipping a production ML compiler.
compiler.py, benchmark.py, test_compiler.py, two graph JSONs, measured result JSONs, generated before/after C, vectorization evidence and test output. All authored source is original. NumPy and GCC are external dependencies, not bundled.
After scalar folding, replace duplicate ordered expressions with an earlier node and rewrite uses, including the output alias. Constants use their float32 bit pattern as the key, so +0 and -0 do not merge. Input identities remain separate. Then perform dead-code elimination and lowering. No algebraic reassociation, commutation, fast-math, automatic fusion selection or profitability model is added. GCC may already eliminate duplicates inside a fused loop; this pass primarily removes redundant IR work and unfused materializations.
two_branch.json represents two squared distances of a shared affine feature to different centers: (0.8*x - 0.3*y + 0.375)^2 + (0.8*x - 0.3*y - 0.2)^2. Its independently expressed branches repeat three affine operations. CSE merges those three nodes: 11 unfused loops and 10 work arrays become eight loops and seven arrays. This is a constructed microbenchmark with intentional shared expressions, not a real imported model.
21 samples per variant, five warmups, all four combinations of CSE on/off and fused/unfused, rotating variant order, preallocated reused buffers. Fixed per-sample repetitions: 2000 / 50 / 3 at 1k / 65k / 1m. Median per call including Python/ctypes overhead, excluding compile/allocation. Raw samples, metadata and a full repeat are included. All four outputs exactly matched the independent NumPy interpreter in these runs. Shared-host noise affects close comparisons; no confidence interval or hardware-counter study is claimed.
| Workload | Elements | Unfused no CSE ms | Unfused CSE ms | CSE-only speedup | Fused CSE ms |
|---|---|---|---|---|---|
| polynomial.json | 1,024 | 0.002137 | 0.002033 | 1.051x | 0.001196 |
| polynomial.json | 65,536 | 0.124769 | 0.123912 | 1.007x | 0.026972 |
| polynomial.json | 1,048,576 | 5.903668 | 6.090944 | 0.969x | 0.671176 |
| model.json | 1,024 | 0.002046 | 0.002047 | 0.999x | 0.002123 |
| model.json | 65,536 | 0.130113 | 0.129463 | 1.005x | 0.348974 |
| model.json | 1,048,576 | 6.160015 | 6.277973 | 0.981x | 5.967970 |
| two_branch.json | 1,024 | 0.002247 | 0.001921 | 1.170x | 0.001138 |
| two_branch.json | 65,536 | 0.161298 | 0.109874 | 1.468x | 0.022255 |
| two_branch.json | 1,048,576 | 7.031299 | 6.312569 | 1.114x | 0.662195 |
CSE-only speedups on the two-branch repeat: 1.170x, 1.519x, 1.200x. No useful work is removed on the other two graphs; their timing changes illustrate measurement noise, not a general CSE benefit. ReLU fusion still loses at 65k.
20 unittest methods pass. The new differential test generates 40 deterministic DAGs with 18 random operations each, adds a vector output, and checks four compiler variants against the original graph interpreter at lengths 0, 1, 17 and 257. Targeted cases also cover lengths 3 and 4099, rewritten duplicate output aliases, direct-input outputs, preserved dead-input ABI, no commutation, signed-zero constant keys, malformed output types, dead-code cycles and constant-fold overflow. Output canaries check one float before and after the requested range; this is not a memory-sanitizer proof. No NaN/Inf or arbitrary-pointer safety claim is added. The malformed-output validation bug is fixed to raise GraphError instead of leaking a TypeError.