macOS: Qwen-Image 2.1 Turbo (text to image, 1-5 image edits) on an MLX-Serve worker

deploy/profiles/mac-apple-silicon.json

@@ -14,7 +14,8 @@
  "env": {
   "LR_BIN": "$REPO/zig-out/bin/localrouter",
   "DEVELOPER_DIR": "${DEVELOPER_DIR:-/Applications/Xcode.app/Contents/Developer}",
-  "STEPS": "bash $REPO/tools/install/steps.sh"
+  "STEPS": "bash $REPO/tools/install/steps.sh",
+  "IMAGE_ROOT": "$J/image-mac"
  },
  "prereqs": [
   {
@@ -173,6 +174,39 @@
      "fix": "network: run again (the download resumes). \"do not match SHA256SUMS\": delete $J/models/decide/knowline-4b-gen5.5 and run again"
     }
    ]
+  },
+  {
+   "id": "qwen-image-2.1-turbo",
+   "key": "image",
+   "title": "Qwen-Image 2.1 Turbo text to image and edits of up to five images (MLX-Serve 4-bit, 8 steps; a cmd worker, loaded on demand)",
+   "licence": {
+    "name": "Qwen Research License: non-commercial research or evaluation only",
+    "url": "https://huggingface.co/Qwen/Qwen-Image-2.1/blob/main/LICENSE"
+   },
+   "accept": [
+    "the Qwen-Image 2.1 Turbo licence (link below)"
+   ],
+   "pins": "tools/image/qwen-mlx/pins-macos-q4.json (4-bit, installed), tools/image/qwen-mlx/pins-macos.json (the MLX-Serve binary; the 8-bit alternative), tools/image/qwen-mlx/requirements-macos.txt",
+   "memory_bytes": 10200547328,
+   "working_bytes": 6979321856,
+   "download_bytes": 10628459115,
+   "disk_bytes": 11500000000,
+   "warm": false,
+   "tool": "tools/image/qwen-mlx/config.example.json: cmd $IMAGE_ROOT/venv/bin/python -I $REPO/tools/image/qwen-mlx/qwen_mlx_worker.py, weights $IMAGE_ROOT/qwen-image-2.1-turbo-mlx-q4, options pack q4 and reference_working_bytes",
+   "skip": "leave image out of --models; picture requests then fail with 'no image model'",
+   "steps": [
+    {
+     "id": "fetch",
+     "title": "the MLX-Serve 26.10.3 binary, the 4-bit Qwen-Image 2.1 Turbo pack (10.5 GB, every file against its pin) and the worker's Python",
+     "download_bytes": 10628459115,
+     "disk_bytes": 11500000000,
+     "needs_accept": true,
+     "run": "bash $REPO/tools/image/qwen-mlx/fetch.sh $IMAGE_ROOT --pack q4",
+     "done": "[ -x $IMAGE_ROOT/venv/bin/python ] && $IMAGE_ROOT/mlx-serve-macos-arm64/mlx-serve --version 2> /dev/null | grep -qx 'mlx-serve 26.10.3' && python3 -I $REPO/tools/package/verify_assets.py --quick $REPO/tools/image/qwen-mlx/pins-macos-q4.json $IMAGE_ROOT/qwen-image-2.1-turbo-mlx-q4",
+     "fix": "network: run again (finished files are kept, a partial one resumes). 'sha256 … pinned' or 'differs from': delete the named file and run again"
+    }
+   ],
+   "reference_working_bytes": 2147483648
   }
  ]
 }

docs/dev/IMAGE-EDIT.md

@@ -218,3 +218,104 @@ which differs between CUDA 13.0 (the box's nvcc, which builds the engine) and 13
 four GELU ops differed by a few bytes in 66,709 and the chained replay diverged from there. The same header serves the
 H3 vision tower. References are resized to about one megapixel with sides up to 2048 (4:1), independent of the
 output's `max_side`; the worker's memory is 38.2 / 41.3 / 44.1 GiB at `edit_refs` 1 / 2 / 3 (31.2 GiB without the pack).
+
+## macOS (Apple Silicon): MLX-Serve worker
+
+On a Mac the image tool is a `cmd` worker, `tools/image/qwen-mlx/qwen_mlx_worker.py`, not the Zig/CUDA engine above. It
+starts MLX-Serve (a native Zig/MLX server) on 127.0.0.1 and is its client, the "model's own server" shape of
+docs/ARCHITECTURE.md. Text to image and edits with one to five input images, the same checkpoint (Qwen-Image 2.1
+Turbo, its own unshifted eight-node `sample_sigmas` schedule).
+
+| Part | Pin (`tools/image/qwen-mlx/pins-macos.json`) |
+| --- | --- |
+| Runtime | MLX-Serve 26.10.3 release binary `mlx-serve-bin-macos-arm64.tar.gz`, sha256 `d1781c89…652b` (source `046b6476`); 26.10.2 is the first that runs Turbo's own schedule, so the worker refuses older |
+| Weights | `ddalcu/Qwen-Image-2.1-Turbo-MLX-Serve-8bit@214ca346`, 17.7 GB: MLX affine 8-bit (group 64) DiT and text-encoder linears, dense f32 VAE, the Qwen3-VL vision tower; every file's size and digest checked by `verify_assets.py` |
+| 4-bit weights (the Mac profile's) | `ddalcu/Qwen-Image-2.1-Turbo-MLX-Serve-4bit@394f9477`, 10.5 GB (`pins-macos-q4.json`): 4-bit DiT and text-encoder linears, the same tower, VAE and schedule; `fetch.sh ROOT --pack q4`, option `pack: "q4"`. The profile and `config.example.json` install and run this pack; its bookings are below |
+| Worker Python | Pillow 13.0.0 (`requirements-macos.txt`, hashes required) |
+
+Setup: `tools/image/qwen-mlx/fetch.sh ROOT --pack q4` (binary, pack, venv; without `--pack` the 8-bit one), then a tool like
+`tools/image/qwen-mlx/config.example.json`. Tests without a model: `ROOT/venv/bin/python -I
+tools/image/qwen-mlx/test_worker.py` (lifecycle, memory, refusals, cancellation, failures, outputs, against a fake
+MLX-Serve).
+
+**Licence.** The weights are Qwen's, under the Qwen Research License Agreement: non-commercial research or evaluation
+only; commercial use needs a separate licence from Qwen. The 8-bit conversion does not change that. MLX-Serve is MIT,
+its bundled MLX libraries Apache-2.0.
+
+**Same request, same meaning.** The worker applies the Spark tool's rules itself (the daemon does not run a `cmd`
+tool's `check`): `max_side` / `max_pixels` on the output, steps 0 or 8, `n` images with seeds `seed`, `seed+1`, …,
+up to `edit_refs` (at most 5) input images in order (the first is the one edited), an edit without a size gets the last
+image's shape at about one megapixel within the limits (`editSize`), inputs under 32 pixels a side or past 4:1 are
+refused, and a mask (alpha 0 where the edit goes) keeps the first image where it is opaque (the `Keep` composite, done
+with Pillow: premultiplied Lanczos, then `Image.composite`; at the soft edges its rounding may differ from the Zig
+blend by 1). `guidance` and `negative_prompt` are not used, as on the Spark. Differences, refused rather than changed
+quietly: an explicit edit size must be a multiple of 32 (MLX-Serve floors it); WebP inputs are sent as lossless PNG
+(MLX-Serve decodes PNG and JPEG). Output is RGBA (the VAE's own alpha, MLX-Serve's `transparent`), with `prompt` and
+`seed` text chunks; the daemon adds the machine-generated label. Cancel: SIGUSR1 closes the stream, MLX-Serve stops at
+its next step and stays loaded. A dead server takes the worker with it (a 503; the next request reloads).
+
+**Numerics.** This is a deliberate numerical change from the Spark engine (8-bit MLX weights, MLX kernels), not a port
+of it: no bit-parity with the Spark or the twin is claimed. The Mac reference is the baseline below, MLX-Serve 26.10.3
+with its lossy `--w8a8` (int8 x int8 DiT linears, on by default on M5 with 64 GiB or more) turned **off**
+(option `w8a8`, default false).
+
+### Baseline (2026-10-30, Mac Studio M5 Ultra, 96 GiB, macOS 27.2)
+
+`tools/image/qwen-mlx/baseline.py ROOT OUT` drives the worker through the protocol as the daemon does, under the
+machine's GPU lock. Its inputs are made first by the same model (so nothing comes from outside): `woman` 1024x1024 seed
+11, `hat` 768x768 seed 12, `cup` 1024x768 seed 13, `dog` 768x1024 seed 14, `room` 1344x768 seed 15 (prompts in the
+script). Every case runs with seeds 7 and 8; edits leave the size to the model (the last input's shape: 1024x1024, or
+1344x768 for five inputs). Memory: the server's resident set sampled every 100 ms (unified memory, MLX's buffers are in
+it; MLX's own `/props` peak agreed within 0.1 GB).
+
+| Case | Inputs | Seconds a image (seed 7 / 8) | Peak resident |
+| --- | --- | --- | --- |
+| load (server start, model, a 256x256 warm-up image) | | 2.3 (weights in the file cache; 11.9 s server start cold) | 16.9 GB resident |
+| text to image 1024x1024 | | 8.48 / 8.50 | 16.3 GB |
+| edit "make it night time…" | woman | 12.92 / 11.26 | 16.8 GB |
+| edit "…wearing the red beanie from image 2" | woman, hat | 14.25 / 14.22 | 16.6 GB |
+| edit "…sitting in the armchair of the room in image 5…" | all five | 25.59 / 24.44 | 17.3 GB |
+| masked edit "give her a bright yellow sweater" | woman + mask | 11.39 / 11.40 | 16.7 GB |
+| exit | | 1.9 | |
+
+Peak over the run 17.33 GB; an 8-bit tool books 16 GiB resident plus 2 GiB working. For comparison the Spark (GB10, FP8)
+makes the 1024x1024 image in 6.3 s and edits one input in 11 to 12.5 s.
+
+**Bookings (the 4-bit pack).** The scheduler books an image job at `resident_bytes + working_bytes`, and an edit at
+`working_bytes + reference_working_bytes × inputs` on top of the resident set (`reference_working_bytes` is an option
+the daemon reads; the worker leaves it alone). The integration run measured the 4-bit pack under load: 9.27 GiB
+resident after the load, a 15.1 GiB peak for a 1-megapixel image, 13.2 GiB for an edit of one input and 24.3 GiB for
+one of five. The profile and `config.example.json` book it at:
+
+| Field | Bytes | |
+| --- | --- | --- |
+| `resident_bytes` | 10200547328 | 9.5 GiB (9.27 measured at load) |
+| `working_bytes` | 6979321856 | 6.5 GiB (9.5 + 6.5 = 16 GiB against the 15.1 GiB peak of a 1 MP image) |
+| `options.reference_working_bytes` | 2147483648 | 2 GiB an input: 18 GiB booked for a 1-input edit (13.2 measured), 26 GiB for 5 inputs (24.3 measured) |
+
+On the Mac profile the image model loads on demand (`warm: false`, `keep_loaded: false`): chat, Drama 3 and Whisper
+stay warm (about 66 of the GPU's 77.8 GiB), and the 8-bit pack's 18 GiB does not fit beside them, so the profile
+installs the 4-bit pack. Still to do: the 4-bit pack against the 8-bit baseline above on quality and speed.
+
+**Why MLX-Serve and not mflux.** Same Turbo checkpoint at 8 bits (`mflux-community/qwen-image-2-1-turbo-mflux-q8@19d2068a`,
+which also has the vision tower), mflux `5171ae15` `QwenImage21Edit`, the same prompts, inputs and seeds, one load:
+
+| | MLX-Serve 26.10.3 | mflux 5171ae15 |
+| --- | --- | --- |
+| text to image 1024x1024 (s, seed 7 / 8) | 8.48 / 8.50 | 11.13 / 8.33 |
+| edit, 1 input | 12.92 / 11.26 | 44.01 / 52.89 |
+| edit, 2 inputs | 14.25 / 14.22 | 32.31 / 25.43 |
+| edit, 5 inputs | 25.59 / 24.44 | 36.46 / 39.11 |
+| peak memory (MLX) | 16.8 to 17.3 GB | 29.1 to 34.0 GB |
+
+**Through the daemon** (the same Mac, LocalRouter built with `zig build -Doptimize=ReleaseSafe`, the tool of
+`config.example.json`; seconds are the HTTP round trip): `/v1/images/generations` 1024x1024 10.8 s for the first
+request (loading the tool included), `n` 2 16.8 s; `/v1/images/edits` with 1, 2 and 5 inputs 11.3, 14.0 and 24.0 s, a
+mask 11.2 s; MCP `generate_image` 8.6 s, `edit_image` with 1 and 2 images 11.3 and 14.5 s. Refused with a 400 and
+the reason: six inputs (the API), steps 25, 1600x1024, and an edit at 1008x1008 (the worker). Outputs carry the
+machine-generated label and the `prompt` / `seed` chunks; the scheduler measured 16,089 MiB resident. A SIGKILLed
+daemon leaves no worker or MLX-Serve behind (`localrouter guard`).
+
+Both produce correct edits with all five inputs. MLX-Serve is as fast for text to image, two to four times faster for
+edits, and needs half the memory; it is also a server that cancels at the next step, where mflux is a library a worker
+would have to host itself.

tools/image/qwen-mlx/baseline.py

@@ -0,0 +1,173 @@
+#!/usr/bin/env python3
+"""The macOS Qwen-Image baseline: the real model through the worker, as the daemon drives it, with times and memory.
+
+    ROOT/venv/bin/python -I tools/image/qwen-mlx/baseline.py ROOT OUT [--only t2i,edit1,edit2,edit5,mask] [--w8a8]
+
+ROOT is fetch.sh's; OUT a directory outside the repository (the images are not committed). The worker loads (timed),
+then makes the fixed prompts and seeds below: 1024x1024 text to image, and edits with 1, 2 and 5 input images (the
+inputs are themselves made here, so the baseline needs nothing from outside) and one with a mask. Every case runs
+twice with two seeds. A sampler reads the server's resident set every 100 ms (unified memory: MLX's Metal buffers are
+in it). OUT/baseline.json holds the numbers and OUT/<case>-<seed>.png the images. Hold the machine's GPU lock while it
+runs. docs/dev/IMAGE-EDIT.md, "macOS", has the results.
+"""
+import argparse
+import json
+import os
+import platform
+import shutil
+import subprocess
+import sys
+import threading
+import time
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+
+# The inputs, made first: (name, prompt, width, height, seed). Sizes differ so the edits see mixed shapes.
+INPUTS = [
+    ("woman", "a photo of a smiling woman with short black hair in a plain grey sweater, studio portrait, soft light", 1024, 1024, 11),
+    ("hat", "a red knitted wool beanie with a white pompom, product photo on a white background", 768, 768, 12),
+    ("cup", "a turquoise ceramic coffee cup with steam rising, product photo on a white background", 1024, 768, 13),
+    ("dog", "a golden retriever sitting on grass, looking at the camera, daylight photo", 768, 1024, 14),
+    ("room", "a cosy reading room with a green armchair, a bookshelf and a big window, warm evening light", 1344, 768, 15),
+]
+SEEDS = (7, 8)
+# (case, prompt, inputs, size or None (the last input's shape), mask)
+CASES = [
+    ("t2i", "a red fox standing in fresh snow at golden hour, a wooden sign in the background that reads \"LocalRouter\"", [], (1024, 1024), False),
+    ("edit1", "make it night time: dark blue sky through the window light, moonlight on her face", ["woman"], None, False),
+    ("edit2", "the woman from image 1 wearing the red beanie from image 2, same portrait and lighting", ["woman", "hat"], None, False),
+    ("edit5", "the woman from image 1 wearing the beanie from image 2, sitting in the armchair of the room in image 5, "
+              "holding the cup from image 3, with the dog from image 4 at her feet", ["woman", "hat", "cup", "dog", "room"], None, False),
+    ("mask", "give her a bright yellow sweater", ["woman"], (1024, 1024), True),
+]
+
+
+class Sampler:
+    """The peak resident set of the worker's mlx-serve, sampled every 100 ms; `mark` starts a new window."""
+
+    def __init__(self, worker_pid):
+        self.worker_pid, self.pid = worker_pid, 0
+        self.peak = self.window = 0
+        self.stop = False
+        threading.Thread(target=self.run, daemon=True).start()
+
+    def run(self):
+        while not self.stop:
+            if not self.pid:
+                out = subprocess.run(["pgrep", "-P", str(self.worker_pid), "mlx-serve"], capture_output=True, text=True).stdout.split()
+                self.pid = int(out[0]) if out else 0
+            if self.pid:
+                out = subprocess.run(["ps", "-o", "rss=", "-p", str(self.pid)], capture_output=True, text=True).stdout.strip()
+                rss = int(out or 0) * 1024
+                self.peak, self.window = max(self.peak, rss), max(self.window, rss)
+            time.sleep(0.1)
+
+    def mark(self):
+        w, self.window = self.window, 0
+        return w
+
+
+class Worker:
+    def __init__(self, root, w8a8):
+        tool = {"id": "qwen-image-2.1-turbo-mlx", "kind": "image", "weights": os.path.join(root, "qwen-image-2.1-turbo-mlx-q8"),
+                "options": {"w8a8": w8a8}}
+        self.p = subprocess.Popen([os.path.join(root, "venv", "bin", "python"), "-I", os.path.join(HERE, "qwen_mlx_worker.py")],
+                                  stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True,
+                                  env=dict(os.environ, LOCALROUTER_TOOL=json.dumps(tool)))
+
+    def call(self, obj):
+        self.p.stdin.write(json.dumps(obj) + "\n")
+        self.p.stdin.flush()
+        while True:
+            m = json.loads(self.p.stdout.readline())
+            if "progress" not in m:
+                return m
+
+
+def gb(n):
+    return round(n / 1e9, 2)
+
+
+def main():
+    a = argparse.ArgumentParser(description=__doc__.split("\n")[0])
+    a.add_argument("root")
+    a.add_argument("out")
+    a.add_argument("--only", default="t2i,edit1,edit2,edit5,mask")
+    a.add_argument("--w8a8", action="store_true", help="MLX-Serve's lossy int8 DiT linears (not the baseline)")
+    args = a.parse_args()
+    out = os.path.abspath(args.out)
+    os.makedirs(out, exist_ok=True)
+    only = set(args.only.split(","))
+    report = {"machine": f"{platform.machine()} {platform.mac_ver()[0]}", "w8a8": args.w8a8, "cases": []}
+
+    w = Worker(args.root, args.w8a8)
+    s = Sampler(w.p.pid)
+    t = time.monotonic()
+    r = w.call({"op": "load"})
+    if not r.get("ok"):
+        sys.exit(f"load failed: {r}")
+    report["load_s"] = round(time.monotonic() - t, 2)
+    report["resident_bytes"] = r["resident"]
+    report["load_peak_rss"] = s.mark()
+    print(f"load {report['load_s']} s, resident {gb(r['resident'])} GB", flush=True)
+
+    def run(case, prompt, refs, size, seed, mask=None):
+        job = os.path.join(out, "jobs", f"{case}-{seed}")
+        shutil.rmtree(job, ignore_errors=True)
+        os.makedirs(job)
+        names = []
+        for i, ref in enumerate(refs):
+            shutil.copy(os.path.join(out, f"input-{ref}.png"), os.path.join(job, f"ref_{i}.png"))
+            names.append(f"ref_{i}.png")
+        img = {"prompt": prompt, "n": 1, "seed": seed, "references": names,
+               "width": size[0] if size else 0, "height": size[1] if size else 0}
+        if mask:
+            shutil.copy(mask, os.path.join(job, "mask_0.png"))
+            img["mask"] = "mask_0.png"
+        s.mark()
+        t = time.monotonic()
+        r = w.call({"op": "generate", "id": f"{case}-{seed}", "dir": job, "request": {"image": img}})
+        secs = round(time.monotonic() - t, 2)
+        peak = s.mark()
+        if not r.get("ok"):
+            sys.exit(f"{case} seed {seed}: {r}")
+        return os.path.join(job, r["files"][0]), secs, peak
+
+    if only - {"t2i"}:
+        for name, prompt, width, height, seed in INPUTS:
+            path, secs, peak = run(f"input-{name}", prompt, [], (width, height), seed)
+            shutil.copy(path, os.path.join(out, f"input-{name}.png"))
+            report["cases"].append({"case": f"input-{name}", "size": f"{width}x{height}", "seed": seed, "s": secs, "peak_rss": peak})
+            print(f"input {name} {width}x{height}: {secs} s, peak {gb(peak)} GB", flush=True)
+    for case, prompt, refs, size, masked in CASES:
+        if case not in only:
+            continue
+        mask = None
+        if masked:
+            from PIL import Image  # the worker's venv has it
+            m = Image.new("RGBA", (1024, 1024), (0, 0, 0, 255))
+            m.paste((0, 0, 0, 0), (256, 560, 768, 1024))  # clear over the sweater: only there may it change
+            mask = os.path.join(out, "mask.png")
+            m.save(mask)
+        for seed in SEEDS:
+            path, secs, peak = run(case, prompt, refs, size, seed, mask)
+            dest = os.path.join(out, f"{case}-{seed}.png")
+            shutil.copy(path, dest)
+            report["cases"].append({"case": case, "inputs": refs, "seed": seed, "s": secs, "peak_rss": peak, "file": dest})
+            print(f"{case} seed {seed}: {secs} s, peak {gb(peak)} GB -> {dest}", flush=True)
+
+    t = time.monotonic()
+    w.p.stdin.write(json.dumps({"op": "exit"}) + "\n")  # no reply: the worker stops the server and exits
+    w.p.stdin.flush()
+    w.p.wait(timeout=120)
+    report["exit_s"] = round(time.monotonic() - t, 2)
+    s.stop = True
+    report["peak_rss"] = s.peak
+    shutil.rmtree(os.path.join(out, "jobs"), ignore_errors=True)
+    with open(os.path.join(out, "baseline.json"), "w") as f:
+        json.dump(report, f, indent=1)
+    print(f"peak {gb(s.peak)} GB; {os.path.join(out, 'baseline.json')}")
+
+
+if __name__ == "__main__":
+    main()