diff --git a/.github/verify-patches-applied.sh b/.github/verify-patches-applied.sh index ab02c9ff..5797b41e 100755 --- a/.github/verify-patches-applied.sh +++ b/.github/verify-patches-applied.sh @@ -43,8 +43,11 @@ fail() { [ -f "$STAMP" ] || fail "patch stamp not found: $STAMP — the applier never ran, so the tree is unpatched" # --- 1. every patch on disk is named in the stamp ----------------------------------------------- -# Self-maintaining on purpose: adding a patch file needs no edit here. The stamp's first line is -# the checked-out llama.cpp commit; every other line is " ". +# Self-maintaining on purpose: adding a patch file needs no edit here. The stamp carries metadata +# lines ("head ", "tree ") plus one " " line per patch. +# Count the patch lines by their own shape rather than by subtracting a fixed number of metadata +# lines: that subtraction was "- 1" and silently went stale the moment the applier gained its +# "tree" line, failing every correct build with "the build dir is stale". on_disk=0 for p in "$PATCH_DIR"/*.patch; do [ -e "$p" ] || fail "no *.patch files in $PATCH_DIR" @@ -53,7 +56,7 @@ for p in "$PATCH_DIR"/*.patch; do grep -qF "$name" "$STAMP" || fail "patch '$name' is on disk but absent from the stamp $STAMP" done -in_stamp="$(($(wc -l < "$STAMP") - 1))" +in_stamp="$(grep -cE '^[^[:space:]]+\.(patch|diff)[[:space:]]' "$STAMP" || true)" [ "$in_stamp" -eq "$on_disk" ] \ || fail "stamp lists $in_stamp patch(es) but $on_disk are on disk — the build dir is stale; configure into a fresh one" diff --git a/CLAUDE.md b/CLAUDE.md index c272b180..93acbd5f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b10976** +Current llama.cpp pinned version: **b10988** ## Upgrading CUDA Version @@ -510,7 +510,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b10976 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b10988 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -550,7 +550,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10976`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10988`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1602,7 +1602,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10976`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10988`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 9f336ef4..fd8f8410 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b10976](https://img.shields.io/badge/llama.cpp-%23b10976-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10976) +[![llama.cpp b10988](https://img.shields.io/badge/llama.cpp-%23b10988-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10988) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index 72c88810..00b76ce1 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -732,3 +732,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b10948–b10969 | patches + upstream verification | **All nine patches apply, and none is droppable.** Following the b10933–b10934 precedent the whole set was applied against the **target** tag up front — `git apply -p1` of all nine, in filename order, against a pristine `b10976` worktree, every one clean — and then independently against pristine `b10969` so this intermediate commit is a valid bisect point rather than an untested waypoint. Two patch targets move in this chunk and both are harmless: `src/llama-model.cpp` (`0012`) gains three additive lines at the `LLM_ARCH_MAPLE` dispatch and rope-type switch, far from the `load_tensors` split arithmetic the patch rewrites, and `tests/CMakeLists.txt` (`0012`) *loses* the `test-peg-parser` PCH line near the peg-parser block, nowhere near the patch's `llama_build_and_test(test-model-split.cpp)` registration. Everything else the nine patches touch — `common/arg.{cpp,h}`, `common/peg-parser.cpp`, all of `tools/server/*.cpp`, and the ~34 standalone `main()` call sites — is **untouched by the entire b10948→b10976 range**. **#28776 is the direct sequel to the previous bump's `0013` drop**: having merged this project's #28775, upstream reverted their temporary in-tree carry of it and added a **non-VXE s390x build to their own CI** (`.github/workflows/build-ibm.yml`), plus the one `UNUSED(nb)` in `ggml/src/ggml-cpu/arch/s390/quants.c` that the scalar path exposed — so the scalar, `-DGGML_NATIVE=OFF` configuration `build-linux-s390x` builds is now guarded upstream as well as here, which is the outcome the `0013` note in `CLAUDE.md` was written to anticipate. | | b10969–b10976 | 7 commits, **20 KiB** — the final chunk, reaching the target release. Nothing under `common/`, `include/`, `src/` or `tools/server/*.{cpp,h}` at all. **#28771** ("cmake : use `PROJECT_SOURCE_DIR` instead of `CMAKE_SOURCE_DIR`") retargets five path references — `tools/server/CMakeLists.txt` (×2), `app/`, `tools/tuning/`, `examples/eval-callback/` and `tests/` — and extends `examples/test-cmake` to cover consumption as a **subproject**. Backend work: **#28897** CUDA enables `i16`/`i32` for `GGML_OP_DUP` (new `int16_t` arm in `cpy.cu`; `ggml_backend_cuda_device_supports_op` now returns `true` unconditionally for `DUP` instead of excluding those two types), and **#28576** HIP switches flash-attention MMA to **fp32 accumulation on MFMA devices** (`fattn-mma-f16.cuh`: `T_C_VKQ` becomes `tile<16,16,float>`, MFMA gets its own `VKQ_C` extent separate from WMMA, and one `GGML_CUDA_FATTN_MMA_CONFIG_CASE` drops 4→3 warps), with **#28909** relaxing upstream's own HIP spill check for it. The rest is **#28646** (WebUI: stop re-probing a disabled `/tools` endpoint on every message) and upstream CI (**#28911**, **#28936**), plus `docs/ops.md` + `docs/ops/CUDA.csv` regeneration. 22 files, 88 insertions, 42 deletions. | **No project source change and nothing on the priority review list** — the chunk touches no header on it and no implementation behind one. The `CMAKE_SOURCE_DIR` → `PROJECT_SOURCE_DIR` fix is precisely the bug class that bites a **FetchContent subproject** like this one (`CMAKE_SOURCE_DIR` resolves to the *top-level* project — here `llama/` — not llama.cpp's own root), but none of the five files it repairs is processed by this build: `LLAMA_BUILD_TOOLS`/`LLAMA_BUILD_SERVER`/`LLAMA_BUILD_APP` are forced OFF and the server TUs are compiled straight into `jllama`, `tests/` and `examples/` are never added, and `tools/mtmd` — the one subdirectory this build *does* add explicitly — was already correct. So the fix is inert here today and removes a latent trap tomorrow; upstream's new `examples/test-cmake` subproject coverage makes a regression of it their CI's problem rather than this project's. The two backend changes are classifier-scoped and additive in effect: the CUDA `DUP` widening only adds accepted types for `cuda13-*`, and the HIP fp32 accumulation is a numerical-accuracy change confined to AMD **MFMA** hardware, which no GitHub-hosted runner has — the `rocm-*` jobs are build-only, so CI proves it compiles and nothing more, as designed. The WebUI commit is auto-followed: `build-webui` rebuilds the Svelte UI from the pinned `GIT_TAG`, so it needs no action here. | | b10969–b10976 | patches + upstream verification | End of the two-chunk walk, and **the patch set is unchanged at nine** — nothing dropped, nothing refreshed. Verified against the pristine target rather than inferred: `git apply -p1` of all nine, in filename order, into a clean `b10976` worktree, every one clean. **All standing drop-checks still say "still required"**, run against the pristine tag because the fail-loud applier detects "does not apply" but never "upstream already fixed this": `0001` (`common_params_parse_main` 0 occurrences in `b10976:common/arg.h`; the WIN32 `argv = utf8.ptrs.data()` override still at `common/arg.cpp:1282`), `0010` (`{"vocab_type", meta.model_vocab_type}` still uncast at `b10976:tools/server/server-context.cpp:4554`), `0012` (bare `splits[i] /= split_sum;` still at `b10976:src/llama-model.cpp:1493`, no zero-sum guard), `0002` (`params_base.load_progress_callback = load_progress_callback;` still unguarded at `server-context.cpp:1095`), and `0003`/`0006`/`0008` (`get_slot_prompt_similarity`, `llama_server_set_embedded`/`llama_server_attach` and `LLAMA_SERVER_WORKER_CMD` all absent upstream). The whole b10948→b10976 range leaves **every** patch target except `src/llama-model.cpp` and `tests/CMakeLists.txt` byte-identical, and both of those move only at a distance from the patched regions. Verified end-to-end for real: `rm -rf build` then `cmake -B build -DBUILD_TESTING=ON` through the real `FetchContent` path, configure clean, stamp at head `987498f4592a76897863cf53711dce38380c082b` (= `b10976`) with **nine** SHA-256 lines; extraction unchanged at **138 CLI / 57 request / 15 trainer** names; full `cmake --build --config Release` clean with **zero** errors; `ctest` **537/537**; `nm -D` **40** `Java_*` exports, **0** mangled; `mvn -pl llama clean test -Dtest=NativeLibraryLoadSmokeTest` **4/4, 0 skipped** — the check that cross-validates the bumped `LLAMA_CPP_VERSION` constant against the linked `build-info`, needing the `clean` because the constant is inlined into the already-compiled test class. | +| b10976–b10988 | 12 commits, **323 KiB** — and the size is the story, so the chunking decision is recorded rather than taken quietly. The range was walked in **three** commits: `b10976→b10980` (54 KiB), `b10980→b10981` (191 KiB) and `b10981→b10988` (78 KiB). The middle step is the one that breaks the runbook's 100 KiB rule, and it is **irreducible**: it is a *single* upstream commit, **#28638** ("OpenVINO: optimize stateful decode and GPU MoE inference"), and there is no intermediate `b` tag inside one commit, so no smaller step exists to take. 35 of its 37 files are `ggml/src/ggml-openvino/**`; the other two are `ci/run.sh` and `docs/backend/OPENVINO.md`. Across the **whole** range, `ggml/src` accounts for 293 KiB of the 323 — the remainder is `src/models` 4.6 KiB, `tests/` 5.5 KiB (never compiled here), `.github` 14 KiB of upstream's own CI, `docs`/`ci`/`CONTRIBUTING.md` 5.3 KiB, and `ggml/include` **0.3 KiB**. Backend work by vendor: **#28599** Metal FA kernels for HSK=96/HSV=64 (MiniCPM3), **#28881** a generic OpenCL `ssm_scan`, **#27637** OpenCL MoE expert-matmul selection by batch size, **#28105** Vulkan sparse flash attention (5 shaders + `ggml-vulkan.cpp`), **#26308** CUDA row-contiguous `SUM_ROWS`, **#28789** RPC weight-only hash caching. The one non-backend change is **#28934**, pure code motion: `build_arch_graph()` moves below the `graph()` template specializations in `src/models/{dflash,eagle3,t5}.cpp`. 67 files, 2828 insertions, 892 deletions. | **No project source change, and zero files on the review surface — literally zero.** Nothing under `common/`, `include/`, `tools/server/` or `tools/mtmd/` moved at all, so every row of the priority API-compatibility table is vacuously satisfied and the three mechanical `tools/server` contract greps have **no input to compare**: the request-field set, its `set_hard_limits` bounds and the emitted response keys cannot have moved. The three `src/models/*.cpp` files are internal upstream TUs and #28934 moves no signature. **One "safe to skip" header does move and was checked rather than waved past**: `ggml/include/ggml-rpc.h` bumps `RPC_PROTO_MAJOR_VERSION` 6 → 7. It is inert here — `GGML_RPC` appears nowhere in `llama/CMakeLists.txt`, `publish.yml`, `build.sh` or `build.bat`, so `ggml-rpc` is never built and the wire protocol it versions is never spoken. The practical risk of the 191 KiB OpenVINO step is correspondingly narrow: `llama/CMakeLists.txt` routes `GGML_OPENVINO` to the `resources_linux_openvino` / `resources_windows_openvino` **classifier** trees only — it is not in the default JAR — and both `openvino-*` jobs are build-only on GPU-less runners, so "must still compile" is the whole of it, and CI checks that directly. | +| b10976–b10988 | patches + upstream verification | **The patch set is unchanged at nine — nothing dropped, nothing refreshed, and nothing even had to be re-examined.** Not one patch-target file is touched anywhere in the range: `common/arg.{cpp,h}`, `common/peg-parser.cpp`, every `tools/server/*.{cpp,h}`, `src/llama-model.{cpp,h}` and `tests/CMakeLists.txt` are all byte-unchanged across b10976→b10988, verified by diffing those paths explicitly rather than inferred from the aggregate. **All standing drop-checks still say "still required"**, run against the pristine tag because the fail-loud applier detects "does not apply" but never "upstream already fixed this": `0001` (`common_params_parse_main` 0 occurrences in `b10988:common/arg.h`; the WIN32 `argv = utf8.ptrs.data()` override still at `common/arg.cpp:1282`), `0010` (`{"vocab_type", meta.model_vocab_type}` still uncast at `b10988:tools/server/server-context.cpp:4554`), `0012` (bare `splits[i] /= split_sum;` still at `b10988:src/llama-model.cpp:1493`, no zero-sum guard). Verified end-to-end from a fresh configure: `rm -rf build` then `cmake -B build -DBUILD_TESTING=ON` through the real `FetchContent` path, configure clean, stamp at head `9f31776c3773cf03f98535c19b7e6d394af374b4` (= `b10988`) with **nine** SHA-256 lines; extraction unchanged at **138 CLI / 57 request / 15 trainer** names; full `cmake --build --config Release` clean with **zero** errors and zero warnings; `ctest` **551/551**; `nm -D` **40** `Java_*` exports, **0** mangled; `mvn -pl llama clean test -Dtest=NativeLibraryLoadSmokeTest` **4/4, 0 skipped** (the `clean` is load-bearing — the `LLAMA_CPP_VERSION` constant is inlined into the already-compiled test class, so without it the cross-check against the linked `build-info` compares the old value); full `mvn test` **1763/0**; SpotBugs **0**; spotless and `javadoc:jar` clean. **One defect was found by this bump rather than by the range**: `.github/verify-patches-applied.sh` counted the stamp's patch lines as "total lines minus one", which silently went stale when the applier's content oracle added a second metadata line (`tree `) in the next commit of the session that introduced the script. The guard therefore failed on every correct tree and would have redded `C++ Tests` on the next pipeline run; it is fixed in this branch by counting patch lines by their own shape instead of by subtraction. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 0077bd6b..181ae78e 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE) FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b10976 + GIT_TAG b10988 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 2fe79fe9..ea38ab65 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -10,13 +10,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10976"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10988"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b10976-"} — call + * plus the resolved upstream commit, e.g. {@code "b10988-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -24,14 +24,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10976"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10988"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b10976"; + public static final String LLAMA_CPP_VERSION = "b10988"; // Constants holder — not instantiable. private LlamaCppVersion() {}