[jayleaton]

Summary

Stacked on #134 (feat/mac-foundation).

KnowLine-4B Gen5.5 (decision tool, /v1/systemone, MCP decide) on Apple silicon. It uses the same openai_server wiring as the Spark, with a macOS launcher in place of serve.py. Mac-only files; nothing the Spark runs changes.

tools/decide/knowline/
├── serve.py               (Spark, unchanged)
├── README.md              + "## macOS": setup, the numerical change, baseline numbers
└── macos/
    ├── serve_mlx.py       the authors' pinned knowline_server.py + knowline_layout.py (hash-checked),
    │                      backend → mlx-lm Qwen3.5: last-position hidden → tied head → f32 log-softmax
    ├── pins-macos.json    bf16 @779944d2 (served), Q5 @053081f7 (compared, not adopted), runtime versions
    ├── pyproject.toml / uv.lock   mlx 0.32.4, mlx-lm 0.32.2, transformers 5.19.0 (tokenizer), no torch
    ├── fetch.sh           weights → $J/models/decide/knowline-4b-gen5.5 (SHA256SUMS), venv $J/venvs/decide-mlx
    ├── test_serve_mlx.py  lifecycle, refusals, cancellation, failures, footprint on the longest prompt
    ├── corpus.json        50 fixed requests / 54 questions (make_corpus.py writes it)
    └── bench.py, memory.py   run + compare; footprint via proc_pid_rusage

Three Mac-specific fixes in the launcher:

• Precision: mlx-lm folds Qwen3.5's 1 + w RMSNorm gains into bf16, a step of 1/128. That doubled the probability error against float32, so the gains stay in float32 and the norm's output is cast back, as transformers does.
• Load time: HTTPServer.server_bind calls getfqdn("127.0.0.1"), which goes to mDNS on macOS and took 35 of the 37 s the load first measured. The launcher skips that lookup.
• First request: a warm-up forward at load builds the Metal kernels, so the first decision takes 76 ms instead of 2.4 s.

Evidence

M5 Ultra 96 GiB, macOS 27.2, real weights, each run under the shared gpulock.

Before: KnowLine had no Mac path. serve.py hardcodes CUDA, and the TensorFold Metal qwen35 loader only admits the 2B geometry.
After: decisions are served on Metal. The reference is the authors' unmodified knowline_server.py --backend hf on torch 2.13 MPS. No Spark output for this corpus is committed.

vs float32 (transformers)	same top answer	max |Δp| median / p90 / max	KL mean / max
transformers bf16 (the Spark's stack)	54/54	0.0003 / 0.0022 / 0.0132	0.00004 / 0.00047
MLX bf16 (served)	54/54	0.0002 / 0.0044 / 0.0132	0.00005 / 0.00047
MLX bf16, stock mlx-lm norm fold	54/54	0.0004 / 0.0044 / 0.0261	0.00009 / 0.00143
MLX 5-bit Q5	54/54	0.0017 / 0.0223 / 0.0902	0.00220 / 0.02238

• Expected answers: 52/53 in every run. All runs are deterministic across passes.
• Recommendation: bf16. Q5 keeps every top answer, but its probabilities move up to 0.09, about 20× bf16's error. It is no faster (voice route 74 vs 73 ms) and saves only 5 GiB.
• bf16 latency (warm medians): voice route 73 ms; one noul 40 ms; three questions 119 ms; long state read twice 488 ms (4,638 tokens), read once 369 ms.
• Load: 1.3 s to healthy, with the weights in the file cache.
• Memory: 8.45 GiB loaded; peak 9.74 GiB over the corpus and 9.70 GiB on the longest possible prompt (5,990 tokens). Proposed booking is 10,724,056,240 B (peak + 256 MiB).
• test_serve_mlx.py: 7/7 OK (KNOWLINE_WEIGHTS set: includes the real-model memory test).
• fetch.sh was run end to end (SHA256SUMS OK, venv check OK).

Through the daemon. localrouter serve was built on this Mac from feat/mac-foundation, with macos/config.example.json.

systemone #1 (2170 ms, load included): {"model":"knowline-4b","route":"generate_image","p":{"generate_image":0.9976,...},"noul":0.0675,"score":1.0796}
systemone #2 (137 ms): identical answers
mcp decide: {"isError":false,"text":"route: video (0.988)\nmodel knowline-4b, 38 ms"}
refusal: {"error":{"message":"a choice needs criteria: an object of 2 to 255 option -> description","type":"invalid_request_error",...}}
info: keep_loaded knowline-4b: loaded (10227 MiB)   server peak 9.53 GiB   daemon exit 0   no server left

The request named the model jev-latest, the System One alias, and it reached knowline-4b. The profile now lists KnowLine as decide in deploy/profiles/mac-apple-silicon.json.

Installer (rebased on foundation 64bcb59): tools/install/bootstrap.sh --profile mac-apple-silicon --dry-run --models decide exits 0 on macOS. All prereqs are ok, and the plan is host/build done and decide/fetch todo (download 8.7 GiB, disk 9.3 GiB, up to 10.0 GiB on demand). After the rebase, zig build -Doptimize=ReleaseSafe succeeds, and test_serve_mlx.py passes 6 OK with 1 skipped (the real-model memory test is skipped because no weights were loaded this round; it passed earlier on the real weights).

Merge Danger

Door: two-way. These are new files under tools/decide/knowline/macos/ plus a README section; reverting removes them, and there is no persisted state.

Blast Radius: component. Only a Mac profile that names serve_mlx.py uses this; Spark/Linux configs and serve.py are untouched.

🤖 Generated with Claude Code