krishhgg · Oct 6, 2026 · Oct 5, 2026 · Oct 5, 2026 · Oct 5, 2026 · Oct 5, 2026
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
@@ -0,0 +1,230 @@
+# On a v* tag, build both bundles, smoke-test them on macOS 15 and 26 runners, publish the
+# GitHub release and update Formula/sys1rust.rb in krishhgg/homebrew-tap. A vX.Y.Z-rcN tag
+# publishes a prerelease and leaves the tap alone. packaging/RELEASE.md is the checklist.
+# A pull request that changes packaging/ or this file runs the build and smoke jobs only.
+name: release
+
+on:
+  push:
+    tags: ["v*"]
+  pull_request:
+    paths:
+      - "packaging/**"
+      - ".github/workflows/release.yml"
+
+permissions:
+  contents: read
+
+concurrency:
+  group: release-${{ github.ref }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+defaults:
+  run:
+    # Runs bash with -eo pipefail, so a failed command inside a pipe fails the step.
+    shell: bash
+
+jobs:
+  build:
+    runs-on: macos-26
+    timeout-minutes: 90
+    steps:
+      - uses: actions/checkout@v7
+      - name: Check the tag against the workspace version
+        if: startsWith(github.ref, 'refs/tags/v')
+        run: |
+          # build.sh reads the version with the same sed, which matches only the
+          # [workspace.package] line. vX.Y.Z publishes that version and vX.Y.Z-rcN a prerelease of it.
+          version=$(sed -n 's/^version = "\(.*\)"$/\1/p' runtime/Cargo.toml)
+          re='^v([0-9]+\.[0-9]+\.[0-9]+)(-rc[0-9]+)?$'
+          if ! [[ $GITHUB_REF_NAME =~ $re ]] || [ "${BASH_REMATCH[1]}" != "$version" ]; then
+            echo "::error::Tag $GITHUB_REF_NAME does not match the workspace version $version. Tag v$version or v$version-rcN."
+            exit 1
+          fi
+      - name: Install the pinned Rust
+        run: |
+          rust=$(cat packaging/rust-toolchain-version)
+          rustup toolchain install "$rust" --profile minimal
+          RUSTUP_TOOLCHAIN=$rust rustc --version
+          {
+            echo "RUSTUP_TOOLCHAIN=$rust"
+            echo "CARGO_HOME=${CARGO_HOME:-$HOME/.cargo}"
+          } >> "$GITHUB_ENV"
+      - name: Cache the cargo registry
+        uses: actions/cache@v6
+        with:
+          path: |
+            ${{ env.CARGO_HOME }}/registry/index
+            ${{ env.CARGO_HOME }}/registry/cache
+          key: cargo-registry-${{ hashFiles('runtime/Cargo.lock') }}
+          restore-keys: cargo-registry-
+      - name: Build the macOS 26 bundle
+        run: packaging/build.sh macos26 dist
+      - name: Build the macOS 14 bundle
+        run: packaging/build.sh macos14 dist
+      - uses: actions/upload-artifact@v7
+        with:
+          name: bundles
+          path: dist/*.tar.gz
+          if-no-files-found: error
+          # The tarballs are gzip files already.
+          compression-level: 0
+
+  smoke:
+    needs: build
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          # macos-15 runs the macOS 14 bundle on an OS older than 26.2, the case that bundle exists for.
+          - { runner: macos-15, flavor: macos14 }
+          - { runner: macos-26, flavor: macos26 }
+    runs-on: ${{ matrix.runner }}
+    timeout-minutes: 45
+    env:
+      FLAVOR: ${{ matrix.flavor }}
+      HF_HUB_CACHE: ${{ github.workspace }}/.hf-cache
+    steps:
+      - uses: actions/checkout@v7
+      - uses: actions/download-artifact@v8
+        with:
+          name: bundles
+          path: dist
+      - name: Unpack and run through a symlink, as Homebrew does
+        run: |
+          mkdir -p "$RUNNER_TEMP/pkg" "$RUNNER_TEMP/bin"
+          tar -xzf dist/sys1rust-*-"$FLAVOR"-arm64.tar.gz -C "$RUNNER_TEMP/pkg"
+          ln -s "$RUNNER_TEMP"/pkg/sys1rust-*/bin/sys1rust "$RUNNER_TEMP/bin/sys1rust"
+          echo "$RUNNER_TEMP/bin" >> "$GITHUB_PATH"
+      - name: Version and model list
+        run: sw_vers && sys1rust --version && sys1rust models
+      - name: Cache the typed-decisions snapshot
+        uses: actions/cache@v6
+        with:
+          path: .hf-cache
+          # The typed-decisions revision in bench/models.lock.json, which sys1rust pins.
+          key: hf-typed-decisions-1a793eb568e6718f15941d08f85432581df534e3
+      - name: Pull typed-decisions
+        run: sys1rust pull typed-decisions
+      - name: Smoke test
+        run: python3 packaging/smoke.py "$(command -v sys1rust)"
+
+  publish:
+    if: startsWith(github.ref, 'refs/tags/v')
+    needs: smoke
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    # Every tag's publish job joins this one group, so 2 tags never write the tap at the same
+    # time. GitHub runs 1 job of a group and holds 1 more. A third tag's job cancels the held
+    # one, and that tag's publish job then needs a rerun.
+    concurrency:
+      group: tap-update
+      cancel-in-progress: false
+    permissions:
+      contents: write
+    env:
+      GH_REPO: ${{ github.repository }}
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          # gh reads GH_TOKEN and the tap clone has its own helper, so git keeps no token here.
+          persist-credentials: false
+      - uses: actions/download-artifact@v8
+        with:
+          name: bundles
+          path: dist
+      - name: Checksums
+        run: cd dist && sha256sum sys1rust-*.tar.gz > SHA256SUMS && cat SHA256SUMS
+      - name: Clone the Homebrew tap and check that TAP_TOKEN can push to it
+        if: ${{ !contains(github.ref_name, '-rc') }}
+        env:
+          TAP_TOKEN: ${{ secrets.TAP_TOKEN }}
+        run: |
+          if [ -z "$TAP_TOKEN" ]; then
+            echo "::error::The TAP_TOKEN secret is empty. Set it to a token that can write to krishhgg/homebrew-tap."
+            exit 1
+          fi
+          # Git asks this credential helper for the password, and the helper reads TAP_TOKEN from
+          # the environment. The token stays out of the clone URL, .git/config, every command line
+          # and the log. The empty helper first drops any helper the runner set up.
+          # shellcheck disable=SC2016
+          git clone --depth 1 -c credential.helper= \
+            -c 'credential.helper=!f() { echo username=x-access-token; echo "password=$TAP_TOKEN"; }; f' \
+            https://github.com/krishhgg/homebrew-tap.git "$RUNNER_TEMP/tap"
+          # The tap is public, so the clone runs without the token. A dry-run push asks GitHub for
+          # git-receive-pack, the service behind every push, and GitHub serves it only to a token
+          # that it accepts and that has push access to the tap. An expired, revoked or read-only
+          # token therefore stops the job here, before the release exists. The dry run sends no
+          # commits, so it does not check branch rules on main. Unlike HEAD, --all also works on
+          # a tap with no commits.
+          git -C "$RUNNER_TEMP/tap" push --dry-run --all origin
+      - name: GitHub release
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          tag=$GITHUB_REF_NAME
+          # gh release create makes a draft, uploads the files and then publishes the draft, so a
+          # cancelled attempt can leave a draft behind, and gh release create would add a second
+          # one. The job stops on a draft of this tag and leaves it to a maintainer. Deleting it
+          # here could remove a live release if someone publishes the draft in between.
+          drafts=$(gh api "repos/$GH_REPO/releases" --paginate \
+            --jq ".[] | select(.draft and .tag_name == \"$tag\") | .html_url" |
+            paste -sd ' ' -)
+          if [ -n "$drafts" ]; then
+            echo "::error::Release $tag has a draft that an earlier attempt left: $drafts. Delete it on the Releases page (or \`gh release delete $tag\` after checking it is still a draft), then rerun the publish job."
+            exit 1
+          fi
+          # This endpoint finds only a published release. A rerun goes on with it only if it has
+          # both bundles and the SHA256SUMS of this run, so the formula's hashes match what users
+          # download.
+          if gh api "repos/$GH_REPO/releases/tags/$tag" --silent 2> /dev/null; then
+            pub=$RUNNER_TEMP/published
+            if ! gh release download "$tag" --dir "$pub" --pattern 'sys1rust-*.tar.gz' --pattern SHA256SUMS ||
+              ! cmp -s dist/SHA256SUMS "$pub/SHA256SUMS" ||
+              ! (cd "$pub" && sha256sum --quiet --check SHA256SUMS); then
+              echo "::error::Release $tag is published with other files than this run built. Rerun only the failed jobs, or delete the release first."
+              exit 1
+            fi
+            echo "Release $tag is published with these bundles and SHA256SUMS"
+            exit 0
+          fi
+          flags=()
+          case "$tag" in *-rc*) flags+=(--prerelease) ;; esac
+          gh release create "$tag" dist/sys1rust-*.tar.gz dist/SHA256SUMS --verify-tag \
+            --title "sys1rust ${tag#v}" --generate-notes "${flags[@]}"
+      - name: Update the Homebrew formula
+        if: ${{ !contains(github.ref_name, '-rc') }}
+        env:
+          TAP_TOKEN: ${{ secrets.TAP_TOKEN }}
+        run: |
+          version=${GITHUB_REF_NAME#v}
+          formula=$RUNNER_TEMP/tap/Formula/sys1rust.rb
+          # A rerun of an older tag's publish job must not put its formula over a newer one. The
+          # tap-update group keeps a second tag's job from changing the tap after the clone.
+          if [ -f "$formula" ]; then
+            current=$(sed -n 's/^ *version "\(.*\)"$/\1/p' "$formula")
+            if ! [[ $current =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
+              echo "::error::The tap's Formula/sys1rust.rb needs 1 version line with an X.Y.Z version, and it has '$current'."
+              exit 1
+            fi
+            if [ "$current" != "$version" ] &&
+              [ "$(printf '%s\n' "$current" "$version" | sort -V | tail -n 1)" = "$current" ]; then
+              echo "::notice::The tap has sys1rust $current, which is newer than $version, so the tap stays as it is. Release $GITHUB_REF_NAME is published."
+              exit 0
+            fi
+          fi
+          sha() { awk -v f="sys1rust-$version-$1-arm64.tar.gz" '$2 == f {print $1}' dist/SHA256SUMS; }
+          mkdir -p "$RUNNER_TEMP/tap/Formula"
+          packaging/render_formula.sh "$GITHUB_REF_NAME" "$(sha macos26)" "$(sha macos14)" > "$formula"
+          cd "$RUNNER_TEMP/tap"
+          git add Formula/sys1rust.rb
+          # A rerun after a successful push finds the formula unchanged. That is the wanted state,
+          # so the step succeeds without a commit.
+          if git diff --cached --quiet; then
+            echo "Formula/sys1rust.rb already describes $GITHUB_REF_NAME"
+            exit 0
+          fi
+          git -c user.name="github-actions[bot]" \
+            -c user.email="41898282+github-actions[bot]@users.noreply.github.com" \
+            commit -m "sys1rust $version"
+          git push origin HEAD
diff --git a/.gitignore b/.gitignore
@@ -19,3 +19,7 @@ bench/contenders/*/src/
 
 # Raw measurement data stays local; results/*.md summarize it
 results/raw/
+
+# Release bundle work dirs and output (packaging/build.sh)
+packaging/.work/
+dist/
diff --git a/packaging/RELEASE.md b/packaging/RELEASE.md
@@ -0,0 +1,76 @@
+# Releasing sys1rust
+
+Pushing a `v*` tag runs `.github/workflows/release.yml`. It builds both bundles on a macOS 26 runner, smoke-tests the macOS 14 bundle on a macOS 15 runner and the macOS 26 bundle on a macOS 26 runner, and publishes the GitHub release with `SHA256SUMS`. A final `vX.Y.Z` tag also updates `Formula/sys1rust.rb` in krishhgg/homebrew-tap. A `vX.Y.Z-rcN` tag publishes a prerelease and leaves the tap alone. A pull request that changes `packaging/` or the workflow runs the build and smoke jobs only.
+
+Run every command below from the repository root on a Mac with an M5-class GPU (GPU generation 17 or later) and macOS 26.2 or later. The strict suite's laya-mlx `settings` tests need MLX's NAX gemms and fail on the fallback that older GPUs get, while the smoke tests run on any Apple silicon Mac. Use 1 shell for all the commands, because later steps reuse `version`, `RUSTUP_TOOLCHAIN`, `HF_HUB_CACHE` and `rc`. The tests, the smoke runs and the Homebrew service load a model on the GPU. Run each of them alone, with no other GPU work on the Mac.
+
+1. Install the pinned Rust and select it for this shell. `packaging/build.sh` and `packaging/third_party.py` stop when `rustc` or `cargo` reports another version.
+   ```sh
+   rust=$(cat packaging/rust-toolchain-version)
+   rustup toolchain install "$rust"
+   export RUSTUP_TOOLCHAIN=$rust
+   ```
+2. Set the version in the `[workspace.package]` table of `runtime/Cargo.toml` and update `runtime/Cargo.lock` to match. build.sh builds with `--locked`, so a stale lock file fails the release build.
+   ```sh
+   cargo update --workspace --manifest-path runtime/Cargo.toml
+   ```
+   Merge both files to main. Then switch to main and read the version back.
+   ```sh
+   git switch main && git pull --ff-only
+   version=$(sed -n 's/^version = "\(.*\)"$/\1/p' runtime/Cargo.toml)
+   echo "$version"
+   ```
+3. Build both bundles, run the strict suite against the macOS 26 MLX build, and smoke-test both bundles. The laya-mlx tests do not ask for the pinned revisions. They load the snapshot that a model's `refs/main` names, or else the model's only snapshot, and `sys1rust pull` writes no `refs/main`. So this step pulls the 3 pinned models (846 MB, 678 MB and 846 MB) into a fresh cache under `packaging/.work`, which then holds only the pinned snapshots. `HF_HUB_CACHE` points every later command in this shell at that cache. Every command must pass. The laya-mlx `settings` tests take about 14 minutes.
+   ```sh
+   packaging/build.sh macos26 dist && packaging/build.sh macos14 dist
+   export HF_HUB_CACHE=$PWD/packaging/.work/hf-cache
+   rm -rf "$HF_HUB_CACHE"
+   for m in typed-decisions multilingual english; do
+     packaging/.work/macos26/sys1rust-$version-macos26-arm64/bin/sys1rust pull $m
+   done
+   export MLX_SYS_PREBUILT_DIR=$PWD/packaging/.work/macos26/wheel/mlx
+   cargo test --release --manifest-path runtime/Cargo.toml --workspace
+   for t in settings loading reference; do
+     SYS1_TEST_ALL_CHECKPOINTS=1 cargo test --release --manifest-path runtime/Cargo.toml \
+       -p laya-mlx --test $t -- --ignored --test-threads 1
+   done
+   cargo test --release --manifest-path runtime/Cargo.toml -p sys1rust --test live -- --ignored
+   cargo test --release --manifest-path runtime/Cargo.toml -p sys1rust --test live_pull -- --ignored
+   for fl in macos26 macos14; do
+     python3 packaging/smoke.py packaging/.work/$fl/sys1rust-$version-$fl-arm64/bin/sys1rust
+   done
+   ```
+4. Tag a release candidate and push the tag. `rc` holds the candidate's tag for this step and the next. The workflow builds and smoke-tests the bundles and publishes a prerelease. It leaves the tap alone. Follow the run on the repository's Actions page. If a job fails, merge the fix to main, run `git pull --ff-only`, set `rc` to the next candidate, such as `v$version-rc2`, and run the tag line again.
+   ```sh
+   rc=v$version-rc1
+   git tag "$rc" && git push origin "$rc"
+   ```
+5. Install the candidate through Homebrew from a throwaway local tap. `packaging/render_formula.sh` writes the formula for the prerelease with the hashes from its `SHA256SUMS`. The service does not see this shell's `HF_HUB_CACHE`, so its first start downloads typed-decisions (846 MB) if `~/.cache/huggingface/hub` does not hold it yet. If `curl` never answers, read `$(brew --prefix)/var/log/sys1rust.log`.
+   ```sh
+   sums=$(gh release download "$rc" --repo krishhgg/sys1rust --pattern SHA256SUMS --output -)
+   sha() { printf '%s\n' "$sums" | awk -v f="sys1rust-$version-$1-arm64.tar.gz" '$2 == f {print $1}'; }
+   brew tap-new --no-git local/sys1rust-test
+   packaging/render_formula.sh "$rc" "$(sha macos26)" "$(sha macos14)" \
+     > "$(brew --repository local/sys1rust-test)/Formula/sys1rust.rb"
+   HOMEBREW_NO_AUTO_UPDATE=1 brew install local/sys1rust-test/sys1rust
+   brew test local/sys1rust-test/sys1rust
+   brew services start local/sys1rust-test/sys1rust
+   until curl -fsS 127.0.0.1:8000/health; do sleep 2; done
+   brew services stop local/sys1rust-test/sys1rust
+   brew uninstall sys1rust && brew untap local/sys1rust-test
+   ```
+6. Tag the release and push the tag. The workflow publishes the release and pushes `Formula/sys1rust.rb` to krishhgg/homebrew-tap with the `TAP_TOKEN` secret, a fine-grained token with contents read and write on that repo only. The publish job stops on an empty secret. Before it creates the release, it also runs `git push --dry-run` against the tap with the token. GitHub answers that only for a token that it accepts and that has push access to the tap, so an expired, revoked or read-only token also stops the job before anything goes public. The dry run sends no commits, so it does not check branch rules on the tap's main.
+   ```sh
+   git tag "v$version" && git push origin "v$version"
+   ```
+7. On a clean account, or after `brew uninstall sys1rust`, check that `brew install krishhgg/tap/sys1rust`, `sys1rust serve` and `brew services start sys1rust` work.
+
+## If the tap update fails
+
+If the publish job fails at the token check or at the push because `TAP_TOKEN` expired or lost access, make a new fine-grained token with contents read and write on krishhgg/homebrew-tap only. Store it with the command below, which prompts for the value. Then open the failed run on the Actions page and choose "Re-run failed jobs". If a cancelled attempt left a draft release of the tag, the rerun stops and names it. Delete it on the Releases page, or with `gh release delete <tag>` after checking that it is still a draft, then rerun the publish job. The rerun keeps a published release only if that release's 2 bundles and `SHA256SUMS` match the run's files, and otherwise stops. Then it pushes the formula. If the formula is already up to date, the job finishes without a commit. If the tap already has a newer version, the job leaves the tap alone and says so in a notice. "Re-run all jobs" builds new bundles, and the publish job then stops rather than replace the published files.
+
+```sh
+gh secret set TAP_TOKEN --repo krishhgg/sys1rust
+```
+
+Publish jobs of all tags run one at a time, and GitHub holds at most 1 more waiting. A third tag pushed while 1 publish job runs and another waits cancels the waiting job. Rerun it.