Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/validate_wrapper.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,7 @@ env:
# One row per upstream release a supported llamadart core ships. The row whose
# commit is the submodule pin builds that checkout as is (tools/ci_scope.py).
LLAMADART_UPSTREAMS: >-
{"v0.7.0": "620d2225cc436ab0b001e3200e590e3b286ef9ab",
"v0.8.0": "8642541b255bfb5dc39cd351a178be09013a55d6",
{"v0.8.0": "8642541b255bfb5dc39cd351a178be09013a55d6",
"v0.9.0": "cd959bae99ff4bed22bef2604f45241c09bbb8c8"}

jobs:
Expand Down
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,8 @@ See `docs/v080_qualification.md` for the v0.8.0 candidate, its wrapper gate,
carried-patch status and consumer ABI delta.
See `docs/v080_android_isa_qualification.md` for the v0.8.0/KleidiAI v1.32.0
source pair, its SVE I8MM-gated Q8 kernels and remaining gates.
See `docs/v090_qualification.md` for the v0.9.0 candidate, patches 0003 and
0004 (kept for the older rows, not applied to v0.9.0) and consumer ABI delta,
See `docs/v090_qualification.md` for the v0.9.0 candidate, patch
0004 (kept for the v0.8.0 row, not applied to v0.9.0) and consumer ABI delta,
and `docs/v090_android_isa_qualification.md` for its source pair and ggml
SVE256 repack paths.
See `docs/v080_1_wrapper_rebuild.md` for the `v0.8.0-1` Metal kernel patch and
Expand Down
2 changes: 1 addition & 1 deletion docs/v070_native_contracts.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ with extraction on crashes the same way when it is turned off.
`set_embeddings()` re-reserves only pooled contexts, which is why pooled
EmbeddingGemma passes.

The carried `0003-embedding-mode-scheduler-reserve.patch` marks the scheduler
The carried `0003-embedding-mode-scheduler-reserve.patch` (since retired) marks the scheduler
for reservation whenever embedding extraction changes, including pooling
NONE, so the next decode reserves for the current mode's graph. The
unchanged-mode path is untouched. A debug log provides the mandatory Android
Expand Down
8 changes: 4 additions & 4 deletions docs/v080_qualification.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ v0.7.0 `620d2225cc436ab0b001e3200e590e3b286ef9ab` plus 291. Its native
release identity is the stable tag `v0.8.0`, which `latest` selects.

Native `v0.8.0` was published on 2026-10-30 and the production submodule is
at v0.8.0 (`update-llama-submodule`, 8005b57). `validate_wrapper.yml` keeps a
`v0.7.0` row of every native lane, so patch 0003 stays until that row goes.
at v0.8.0 (`update-llama-submodule`, 8005b57). Patch 0003 and the
`v0.7.0` row of `validate_wrapper.yml` were later retired.

The first scheduled dispatch, on 2026-10-29, failed in every build job
compiling the wrapper against the v0.8.0 `mtmd` API and published nothing; the
Expand Down Expand Up @@ -45,8 +45,8 @@ All three apply strictly to v0.8.0; 0001 and 0002 are applied.
b11870) makes `set_embeddings` re-reserve on every toggle. Its
`upstream_fix` text is that condition, so a v0.8.0 build does not apply it,
its Android marker is not required and `assets.json` does not list it.
v0.7.0 lacks the text and still applies it. Delete the patch once no
`validate_wrapper.yml` row builds v0.7.0 or older.
v0.7.0 lacked the text, so the patch was kept until the v0.7.0 row
went; it has since been deleted.

Evidence, 2026-10-29, Apple M4 Max, macOS arm64, CPU/Accelerate, Metal and
KleidiAI off, Release:
Expand Down
16 changes: 7 additions & 9 deletions docs/v090_qualification.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ Stable upstream v0.9.0, `cd959bae99ff4bed22bef2604f45241c09bbb8c8`
identity is the stable tag `v0.9.0`, which `latest` selects.

This change moves the production submodule to v0.9.0 and adds it as the
pinned `LLAMADART_UPSTREAMS` row of `validate_wrapper.yml`, next to the v0.7.0
and v0.8.0 rows. After the 2026-11-05 automatic dispatch failed at the CPU
ISA audits
pinned `LLAMADART_UPSTREAMS` row of `validate_wrapper.yml`, next to the v0.8.0
row; the v0.7.0 row was retired with patch 0003. After the 2026-11-05
automatic dispatch failed at the CPU ISA audits
([v090_android_isa_qualification.md](v090_android_isa_qualification.md)),
native `v0.9.0` was published on 2026-11-08 from `a604639` by a manual
`auto_native_release.yml` dispatch
Expand Down Expand Up @@ -59,16 +59,14 @@ passes `git apply --check` on both. `series.json` is unchanged.
| --- | --- | --- | --- |
| 0001 Adreno 750 matvec (#79) | Applied | [#29165](https://github.com/ggml-org/llama.cpp/pull/29165) open draft; `use_subgroups` selection unchanged on `master` | Keep |
| 0002 buffer device address (leehack/llamadart#782) | Applied | No fix: `master` still takes `bufferDeviceAddress` from `VkPhysicalDeviceVulkan12Features` without checking the entry point; upstream supports Vulkan 1.2 and later only | Keep |
| 0003 embedding-mode reserve (#114) | Not applied: `upstream_fix` present ([#31086](https://github.com/ggml-org/llama.cpp/pull/31086)) | Fixed since v0.8.0 | Needed only by the v0.7.0 row |
| 0004 Metal unary cast (#154) | Not applied: `upstream_fix` present, `required_text` absent ([#31360](https://github.com/ggml-org/llama.cpp/pull/31360)) | Fixed after v0.8.0 | Needed only by the v0.8.0 row |
| 0005 ARM F16 dot (#174) | Applied | [#31734](https://github.com/ggml-org/llama.cpp/issues/31734) open, no fix; non-SVE NEON `ggml_vec_dot_f16` still accumulates in `float16x8_t` | Keep |
| 0006 ARM F16 GEMM (#174) | Applied | Same issue; llamafile still uses `tinyBLAS<8, float16x8_t, …>` | Keep |

v0.7.0 needs 0001–0003, 0005 and 0006 (0004 does not fit it, since it predates
the regression). v0.8.0 needs 0001, 0002 and 0004–0006. Deleting 0003 means
dropping the v0.7.0 `LLAMADART_UPSTREAMS` row (cores 0.15–0.17); deleting 0004
means dropping the v0.8.0 row (core 0.18). Both are support-policy decisions
for the maintainer.
v0.8.0 needs 0001, 0002 and 0004–0006. Patch 0003 and the v0.7.0
`LLAMADART_UPSTREAMS` row (cores 0.15–0.17) were retired once cores on v0.8.0
shipped; deleting 0004 means dropping the v0.8.0 row (core 0.18), a
support-policy decision for the maintainer.

`check-apple` does not require the 0004 marker on v0.9.0, and
`metal_os_floor.yml` checks the unpatched v0.9.0 Metal kernels on the oldest
Expand Down
12 changes: 0 additions & 12 deletions patches/llama.cpp/0003-embedding-mode-scheduler-reserve.patch

This file was deleted.

3 changes: 0 additions & 3 deletions patches/llama.cpp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,6 @@ these hold:
- The patch has an issue in `leehack/llamadart-native` or `leehack/llamadart`
and, where one exists, an upstream pull request or issue.

The v0.7.0 embedding-mode reservation repair and its exact baseline/candidate
evidence are documented in [v070_native_contracts.md](../../docs/v070_native_contracts.md).

## Exception for native #174

The maintainer approved a narrow exception to the first two eligibility
Expand Down
9 changes: 0 additions & 9 deletions patches/llama.cpp/series.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,6 @@
{"library": "libggml-vulkan.so", "text": "has no vkGetBufferDeviceAddress"}
]
},
{
"file": "0003-embedding-mode-scheduler-reserve.patch",
"tracking": "https://github.com/leehack/llamadart-native/issues/114",
"upstream": "https://github.com/ggml-org/llama.cpp/pull/31086",
"android_markers": [
{"library": "libllama.so", "text": "llamadart re-reserves embedding mode"}
],
"upstream_fix": " if (cparams.embeddings != value) {\n sched_need_reserve = true;\n }\n"
},
{
"file": "0004-metal-unary-cast-older-os-compilers.patch",
"tracking": "https://github.com/leehack/llamadart-native/issues/154",
Expand Down
5 changes: 2 additions & 3 deletions tests/test_ci_scope.py
Original file line number Diff line number Diff line change
Expand Up @@ -112,9 +112,8 @@ def test_every_native_lane_builds_each_upstream_row_once(self):
workflow=(ROOT/'.github/workflows/validate_wrapper.yml').read_text()
table=re.search(r"^ LLAMADART_UPSTREAMS: >-\n((?: .*\n)+)",workflow,re.MULTILINE)[1]
rows=json.loads(table)
# Shipped llamadart cores still load these lines, so they keep their rows.
self.assertLessEqual({'v0.7.0':'620d2225cc436ab0b001e3200e590e3b286ef9ab',
'v0.8.0':'8642541b255bfb5dc39cd351a178be09013a55d6'}.items(),rows.items())
# Shipped llamadart cores still load this line, so it keeps its row.
self.assertLessEqual({'v0.8.0':'8642541b255bfb5dc39cd351a178be09013a55d6'}.items(),rows.items())
upstream_rows(table,pinned_llama_cpp())
self.assertNotIn("'pinned'",workflow)
self.assertEqual(workflow.count('QUALIFICATION_SHA:'),len(JOBS))
Expand Down
36 changes: 14 additions & 22 deletions tests/test_llama_cpp_patches.py
Original file line number Diff line number Diff line change
Expand Up @@ -355,31 +355,23 @@ def test_the_build_applies_the_series_after_adding_llama_cpp(self) -> None:
self.assertLess(added, applied)
self.assertLess(applied, cmake.index("add_library(llamadart_lib SHARED"))

def carried_after(self, context_source: str | None, newline: str = "\n") -> list[str]:
def carried_after(self, unary_source: str | None, newline: str = "\n") -> list[str]:
with tempfile.TemporaryDirectory() as directory:
if context_source is not None:
source = Path(directory) / "src/llama-context.cpp"
source.parent.mkdir()
source.write_bytes(context_source.replace("\n", newline).encode())
if unary_source is not None:
source = Path(directory) / self.UNARY
source.parent.mkdir(parents=True)
source.write_bytes(unary_source.replace("\n", newline).encode())
return [entry["file"] for entry in patches.manifest_entries(
patches.PATCHES_DIR, Path(directory))]

def test_the_embedding_reserve_patch_is_retired_only_where_upstream_has_the_fix(self) -> None:
entry = next(entry for entry in patches.load_series()
if entry["file"] == "0003-embedding-mode-scheduler-reserve.patch")
# v0.8.0 (upstream ggml-org/llama.cpp#31086) toggles embeddings like this.
self.assertEqual(" if (cparams.embeddings != value) {\n"
" sched_need_reserve = true;\n"
" }\n", entry.get("upstream_fix"))
fixed = ("void llama_context::set_embeddings(bool value) {\n"
+ entry["upstream_fix"] + " cparams.embeddings = value;\n}\n")
unfixed = ("void llama_context::set_embeddings(bool value) {\n"
" cparams.embeddings = value;\n}\n")
patch = "0003-embedding-mode-scheduler-reserve.patch"
self.assertNotIn(patch, self.carried_after(fixed))
self.assertNotIn(patch, self.carried_after(fixed, "\r\n"), "a CRLF checkout")
self.assertIn(patch, self.carried_after(unfixed))
self.assertIn(patch, self.carried_after(None), "a missing source retires nothing")
def test_the_metal_patch_is_retired_only_where_upstream_has_the_fix(self) -> None:
fixed = self.metal_patch_entry()["upstream_fix"]
unfixed = "constexpr specialization\n"
self.assertNotIn(self.METAL_PATCH, self.carried_after(fixed))
self.assertNotIn(self.METAL_PATCH, self.carried_after(fixed, "\r\n"), "a CRLF checkout")
self.assertIn(self.METAL_PATCH, self.carried_after(unfixed))
self.assertIn(self.METAL_PATCH, self.carried_after(None),
"a missing source retires nothing")

@unittest.skipUnless((UPSTREAM / "CMakeLists.txt").is_file(),
"third_party/llama.cpp is not checked out")
Expand Down Expand Up @@ -473,7 +465,7 @@ def test_the_manifest_command_reads_the_given_upstream_not_the_submodule(self) -
[sys.executable, str(ROOT / "tools/llama_cpp_patches.py"),
"manifest", "--upstream", directory], capture_output=True, text=True)
self.assertEqual(0, result.returncode, result.stderr)
self.assertEqual(["0001", "0002", "0003", "0004", "0005", "0006"],
self.assertEqual(["0001", "0002", "0004", "0005", "0006"],
[e["file"][:4] for e in json.loads(result.stdout)])

@unittest.skipUnless((UPSTREAM / "CMakeLists.txt").is_file(),
Expand Down
10 changes: 3 additions & 7 deletions tests/test_validate_android_artifacts.py
Original file line number Diff line number Diff line change
Expand Up @@ -104,15 +104,11 @@ def test_tool_accepts_the_shared_runtime_and_names_its_library(self):
self.assertEqual(0, result.returncode, result.stderr)
self.assertIn('__cxa_throw: defined by libggml-base.so', result.stdout)

def test_tool_requires_the_embedding_reservation_repair_until_upstream_has_it(self):
def test_tool_requires_the_f16_dot_repair_marker(self):
result = self.run_tool(shared_runtime(),
omitted_marker='llamadart re-reserves embedding mode')
required = {entry['file'] for entry in patches.required_series()}
if '0003-embedding-mode-scheduler-reserve.patch' not in required:
self.assertEqual(0, result.returncode, result.stderr)
return
omitted_marker='llamadart F16 dot F32 accumulation')
self.assertEqual(1, result.returncode, result.stderr)
self.assertIn('libllama.so: built without 0003-embedding-mode-scheduler-reserve.patch',
self.assertIn('libggml-cpu.so: built without 0005-arm-f16-dot-f32-accumulation.patch',
result.stderr)

def test_tool_rejects_a_second_definition(self):
Expand Down
Loading