Upgrade llama.cpp from b10938 to b10948

.github/workflows/publish.yml

@@ -803,7 +803,8 @@ jobs:
       # exactly right for what it is (a big-endian correctness gate for our own layer, not a perf target).
       # Do NOT "fix" a VXE-related compile error by adding -DGGML_VXE=ON: that define sets __VXE__ and
       # __VXE2__ together while -march stays at the toolchain default arch11, so every z14+ builtin is
-      # rejected. See the patches/0013 row in CLAUDE.md for the measured comparison.
+      # rejected. See the "`0013` was dropped at the b10948 bump" note in CLAUDE.md for the measured
+      # comparison (the patch itself is gone -- upstream merged it as ggml-org/llama.cpp#28775).
       - name: Build libraries (cross-compile s390x)
         shell: bash
         run: |

CLAUDE.md

@@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI.
 
-Current llama.cpp pinned version: **b10938**
+Current llama.cpp pinned version: **b10948**
 
 ## Upgrading CUDA Version
 
@@ -502,7 +502,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi
 ships no UI):
 ```bash
 # needs node/npm + network for the asset build; the embed step is plain cmake -P
-git clone --depth 1 --branch b10938 https://github.com/ggml-org/llama.cpp /tmp/lc
+git clone --depth 1 --branch b10948 https://github.com/ggml-org/llama.cpp /tmp/lc
 ( cd /tmp/lc/tools/ui && npm ci && npm run build )
 mkdir -p webui-generated /tmp/ui-gen
 cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \
@@ -542,7 +542,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend:
 - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored
   as the repo secret **`DEPOT_TOKEN`**.
 
-Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10938`), the
+Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10948`), the
 ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the
 *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's
 per-branch `actions/cache`), so every branch builds incrementally; a `b<nnnn>` version bump
@@ -749,7 +749,33 @@ Current patches:
 | `0011-peg-parser-lenient-invalid-utf8.patch` | **A model that emits one malformed UTF-8 byte turns a finished generation into an HTTP 500.** The server parses *every* completion through `common_chat_parse()`; with no chat parser configured (plain `/completion`) that is the content-only fallback `content(rest()) + end()`, whose scan is `common_peg_until_parser` (`common/peg-parser.cpp`). `common_chat_peg_parse()` always parses in **lenient** mode, and that scan tolerates an `INCOMPLETE` trailing UTF-8 sequence by keeping the text before it — but the `INVALID` branch right below it returns `FAIL` unconditionally, ignoring leniency. One stray byte anywhere in the generated text therefore throws `"The model produced output that does not match the expected Content-only format"` and the request 500s even though generation completed normally (`stop processing: n_tokens = 4, truncated = 0`). The patch makes the `INVALID` branch respect `ctx.is_lenient()` exactly like the `INCOMPLETE` branch — keep the text up to the malformed byte — and adds an upstream `tests/peg-parser/test-unicode.cpp` case pinning both the lenient and the still-failing strict behavior. **Strict mode is unchanged**, which is what keeps upstream's own tests green: `tests/peg-parser/test-unicode.cpp` *does* assert `FAIL` on invalid UTF-8 through the *until* parser (a `malformed UTF-8` block with three `p.until("</tag>")` cases), but each builds a bare `common_peg_parse_context` with no `COMMON_PEG_PARSE_FLAG_LENIENT`, so the lenient-only change cannot reach them. This patch adds its case inside that same block. Found by `NativeServerAttachIntegrationTest.completion_overHttp_served`, which 500s on all six Java CI platforms. Upstream-submittable; **not yet filed upstream**. Touches only `common/peg-parser.cpp` + that test, which no other patch touches, so it is independent of `0001`/`0006`/`0007`. Runnable guard: the `ContentOnlyParseUtf8` tests in `src/test/cpp/test_utils.cpp` — unlike the upstream test they are compiled and run in CI on every platform, so a bump that drops this patch reds `C++ Tests` instead of one Java job. |
 | `0006-server-embed-native-server-jni.patch` | **Makes `server.cpp`'s `llama_server` embeddable in the JVM** so the `NativeServer` JNI bridge can run the full upstream HTTP server (WebUI included) inside `libjllama` — see "Two server modes" below. b9870 already exposes `int llama_server(int, char**)` (non-static; no `main` in the file), so the patch only adds embedded-mode support: (1) a `g_llama_server_embedded` flag + `llama_server_set_embedded()` / `llama_server_request_shutdown()` (declared in the committed `src/main/cpp/native_server_bridge.h`); (2) skips installing the process-wide SIGINT/SIGTERM handlers when embedded (they would hijack the JVM's); (3) in embedded mode parses the **forwarded** argv via `common_params_parse` instead of `common_params_parse_main` (whose `GetCommandLineW` recovery would pick up `java.exe`'s command line — the same Windows class of bug `0001` fixes). `llama_server_request_shutdown()` mirrors the SIGTERM path (invokes the installed `shutdown_handler` → `ctx_server.terminate()` unblocks `start_loop()`), giving JNI an out-of-band stop since `ctx_server` is loop-local. Applies **after `0001`** (which flips this call site to `common_params_parse_main`), so its context is the post-`0001` tree; regenerate against `0001`+source on a bump. Only touches `tools/server/server.cpp`. |
 | `0012-model-guard-zero-split-sum-and-name-the-device-index.patch` | **A GPU that reports zero free memory makes every model load fail with the unactionable `error loading model: vector`.** `llama_model_base::load_tensors` (`src/llama-model.cpp`) weights the per-device layer split by `ggml_backend_dev_memory()`'s `free`, then normalises: `splits[i] /= split_sum`. With a single device reporting `free == 0` that is `0/0` → **NaN** in every split point; NaN compares false against everything, so the `std::upper_bound` below returns the end iterator, `layer_gpu == n_devices()`, and `devices.at(layer_gpu)` throws `std::out_of_range` — whose libc++ `what()` is the bare string `"vector"`, which `llama.cpp`'s `catch (const std::exception &)` prints verbatim. Upstream's `free == 0 && total == 0` host-memory fallback does **not** fire, because `total` is `recommendedMaxWorkingSetSize` and is non-zero. **Reachable since b10618..b10797**: upstream `8c0b9cd04` ("metal : fix memory query under low-memory conditions", [#27701](https://github.com/ggml-org/llama.cpp/pull/27701)) changed `ggml-metal-device.m` to `*free = *total > cur ? *total - cur : 0`; before that clamp an over-committed device (`currentAllocatedSize > recommendedMaxWorkingSetSize`) *underflowed* to a huge `size_t`, which normalised fine, so the same precondition was harmless. That is why the `Java Tests macOS …` jobs went red at the b10792→b10797 step while every Linux/Windows job stayed green — **and why only a GPU build can fail this way at all**: `act_gpu_layers` is `devices.empty() ? 0 : …`, so with no GPU backend `devices` is empty, every layer returns early on `cpu_dev`, and the `.at()` line is unreachable. **Shape:** the two blocks are lifted out of `load_tensors` into free functions declared in `src/llama-model.h`, purely so they can be driven by a test — the failing state needs a real over-committed GPU and cannot be arranged through any public API. `llama_model_splits_normalize()` carries **the fix**: on `split_sum == 0` it `LLAMA_LOG_WARN`s and falls back to an even split (`splits[i] = float(i+1)/splits.size()`), the only neutral choice when no device can be preferred and exactly right for a single device. `llama_model_splits_select_device()` carries **the diagnostic**: it bounds-checks the index and throws a `std::runtime_error` naming the function, the offloaded layer, the device index, the split-point count **and the split points themselves** — with NaN splits that message prints `nan` and names the cause outright, which is precisely what was missing when this had to be diagnosed by reading source. **A second, backend-independent trigger reaches the same line**, found while writing this up and verified against the unfixed library: `--tensor-split` values are parsed with `std::stof` and never range-checked (`common/arg.cpp`), so `-ts 1,-1` cancels out, `split_sum` is 0 again, the split points become `[inf, -nan]`, and every layer maps one past the last device — on CUDA, Vulkan or ROCm just as much as on Metal, with no memory pressure involved. That is what makes this an ordinary upstream defect rather than a Metal edge case, and the warning names both causes rather than only the memory one. Also adds upstream `tests/test-model-split.cpp` (5 cases in upstream's `testing.h` style) + its `llama_build_and_test` registration. Touches `src/llama-model.{cpp,h}`, `tests/test-model-split.cpp` and `tests/CMakeLists.txt` — **none** of which any other patch touches, so it is independent of all of them. Upstream-submittable ("model: fall back to an even split when no device reports free memory"); **not yet filed upstream**. **Runnable guard: `src/test/cpp/test_model_split.cpp`** — a FetchContent subproject builds with `LLAMA_BUILD_TESTS=OFF`, so the upstream test above is applied-but-never-compiled here (same as `0001`'s test). That file drives the same two functions from `jllama_test`, which runs on **every** platform in `C++ Tests`, so a bump that drops this patch fails the build at link time everywhere instead of surfacing as one red macOS Java job. **Verification limit — read before assuming this can be dropped:** the *failing path* still cannot be reached without a GPU backend, so the guard pins the arithmetic (what actually broke), not the end-to-end load; the end-to-end proof is the macOS CI job. On a bump, re-check whether upstream added its own `split_sum == 0` guard (grep `split_sum` in `src/llama-model.cpp`) and **drop this patch rather than refreshing it** if they did — the fail-loud applier detects "does not apply", never "upstream already fixed this". |
-| `0013-s390x-repack-guard-vxe-only-helpers.patch` | **A new upstream file makes the s390x build fail to compile, with nothing in this project involved.** b10902 added `ggml/src/ggml-cpu/arch/s390/repack.cpp` (upstream #28667, s390x q4_0 repack; the file does not exist at b10883). Every *function body* in it is guarded `#if defined(__VXE__) || defined(__VXE2__)`, but three `static inline` helpers — `vxe_dot_acc`, `vxe_splat_granule`, `vxe_fold` — sit at file scope **between** two guarded blocks with no guard of their own, and their signatures name `int16x8_t` / `int8x16_t` / `int32x4_t`, which `ggml-cpu-impl.h` only typedefs *inside* that same guard. So a non-VXE s390x build dies with three `does not name a type` errors before it reaches any of the code it is supposed to skip. The patch wraps the three definitions in the identical guard — all 21 call sites are already inside `__VXE__` blocks, so nothing else moves. **Why this project builds s390x without VXE, which is the half that is ours:** `ggml/CMakeLists.txt` declares `option(GGML_VXE "ggml: enable vxe" ${GGML_NATIVE})` — the VXE default *follows* `GGML_NATIVE`. The `build-linux-s390x` job passes `-DGGML_NATIVE=OFF`, which is correct for a cross-build (it must not bake the x86 build host's `-march=native` into an s390x artifact) but also silently switches VXE off. No `-mvx -mzvector` is then passed, `__VEC__` is undefined, and `ggml-cpu-impl.h`'s `#if defined(__s390x__) && defined(__VEC__)` self-define of `__VXE__`/`__VXE2__` never fires. The job has therefore always produced a **scalar** s390x binary; that was invisible until a file arrived that does not compile that way. **Do not "fix" this with `-DGGML_VXE=ON`** — measured, not assumed: that makes it strictly worse, because the self-define above sets `__VXE__` *and* `__VXE2__` together as soon as `__VEC__` exists, while `-march` stays at the toolchain default `arch11`, so the three errors become dozens of `'__builtin_s390_vec_*' matching variant requires z14 or higher`. The only flag-side alternative is `-DGGML_VXE=ON` **plus** `-march=z15`, which does compile but raises the shipped artifact's hardware floor to z15 and makes the qemu `ctest` gate depend on VXE2 emulation — a real trade for vector kernels this job does not use (it is a big-endian *correctness* gate for our own layer, not a performance target). The patch keeps the configuration that worked through b10883 and changes nothing about the artifact. **Verified with the real cross toolchain** (`s390x-linux-gnu-g++`), not by inspection: unpatched + CI's flags reproduces the three CI errors exactly; patched + CI's flags compiles clean; patched + `-mvx -mzvector -march=z15` also compiles clean, so the patch does not foreclose a future vector build. Touches only that one file, which no other patch touches. Upstream-submittable ("ggml-cpu: guard the VXE-only helpers in the s390x repack path"); **not yet filed upstream**. **On a bump, check whether upstream guarded them itself and DROP this patch rather than refreshing it** — the fail-loud applier detects "does not apply", never "upstream already fixed this". There is no runnable guard for it beyond CI: the file is compiled only for s390x, so `build-linux-s390x` *is* the test, and it fails loudly at compile time. |
+
+**`0013` was dropped at the b10948 bump.** Upstream merged this project's own PR
+[ggml-org/llama.cpp#28775](https://github.com/ggml-org/llama.cpp/pull/28775)
+("ggml-cpu(s390x): guard VXE-only repack helpers", commit `6978052`, first tagged at **b10948**):
+`ggml/src/ggml-cpu/arch/s390/repack.cpp` now wraps `vxe_dot_acc` / `vxe_splat_granule` / `vxe_fold`
+in the same `#if defined(__VXE__) || defined(__VXE2__)` guard the patch added — byte-identical apart
+from the trailing `// __VXE__ || __VXE2__` comment the patch put on the `#endif`. So the non-VXE
+s390x compile break the patch fixed (three `does not name a type` errors before any skipped code is
+reached) no longer exists upstream, and the applier failed loud with "does not apply cleanly" at
+configure time exactly as designed — the guard it wants to add is already there. Dropped, not
+refreshed, per the `0009` precedent.
+
+**The s390x configuration this leaves in place is still ours to know, because it is what made the
+patch necessary and it did not change.** `ggml/CMakeLists.txt` declares
+`option(GGML_VXE "ggml: enable vxe" ${GGML_NATIVE})`, so the `build-linux-s390x` job's
+`-DGGML_NATIVE=OFF` (correct for a cross-build — an x86 host must not bake `-march=native` into an
+s390x artifact) silently switches VXE off too: no `-mvx -mzvector`, `__VEC__` undefined, and
+`ggml-cpu-impl.h`'s `#if defined(__s390x__) && defined(__VEC__)` self-define of `__VXE__`/`__VXE2__`
+never fires. The job therefore ships a **scalar** s390x binary, which is right for what it is (a
+big-endian *correctness* gate for this project's own layer, not a performance target). **Do not
+"fix" a future VXE-related compile error with `-DGGML_VXE=ON`** — measured, not assumed: that
+self-define sets `__VXE__` *and* `__VXE2__` together while `-march` stays at the toolchain default
+`arch11`, so a handful of errors becomes dozens of `'__builtin_s390_vec_*' matching variant requires
+z14 or higher`. The only flag-side alternative is `-DGGML_VXE=ON` **plus** `-march=z15`, which
+compiles but raises the artifact's hardware floor to z15 and makes the qemu `ctest` gate depend on
+VXE2 emulation. If a comparable break resurfaces, re-check upstream's guard against this description
+before reintroducing a local patch.
 
 **`0009` was dropped at the b10280 bump.** Upstream merged
 [sheredom/subprocess.h#104](https://github.com/sheredom/subprocess.h/pull/104) — the exact fix this
@@ -1536,7 +1562,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson"
 
 #### Upstream source location (in CMake build tree)
 
-llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10938`.
+llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10948`.
 
 **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely
 by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the

README.md

@@ -11,7 +11,7 @@
 **Build:**  
 ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational)  
 ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey)  
-[![llama.cpp b10938](https://img.shields.io/badge/llama.cpp-%23b10938-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10938)  
+[![llama.cpp b10948](https://img.shields.io/badge/llama.cpp-%23b10948-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10948)  
 [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/)  
 ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162)  
 [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev)