Skip to content

Checkpoints: an index may name a shard in a folder inside the model folder; shard edits keep plain names - #1179

Closed
Bizuayeu wants to merge 1 commit into
ashhart:mainfrom
Bizuayeu:pr/checkpoint-shard-folders
Closed

Bizuayeu wants to merge 1 commit into
ashhart:mainfrom
Bizuayeu:pr/checkpoint-shard-folders

Conversation

@Bizuayeu

Copy link
Copy Markdown
Contributor

What this changes

An index may map tensors to a shard in a folder inside the model folder. weightMap takes a plain name, or plain names joined by / (a new shardName in core/checkpoint_index.zig). shard_edit's writes (grow and overwrite, through which Living Weights (--slide) and LoRA merges write shards, and bake) check with a new plainWeightMap and keep taking plain names only, and --slide refuses such a checkpoint at startup (native/slide_adapter.zig). RADIO finds its image shards by the index's names rather than by their basenames (families/radio/weights.zig).

1.1.2's shared index check (46ce6df) takes a shard only as a plain file name, so it refuses with BadIndex both checkpoints #1022 sizes. At revision 76348ddb, nvidia/GLM-5.3-Flash-FP4-Local's index maps 214 of its 295,041 tensors to tensors/native-QB.safetensors; nvidia/Qwen3.8-Flash-Next-FP4-Local keeps all of its shards in tensors/, as #1022 notes. #1050 loads the first, so it needs this on 1.1.2.

What stays: no accepted name reaches outside the model folder or into its hidden folders. Each part goes through plainName: it may not be empty, . or .., end in . or a space, or hold a drive, a backslash, a control character or a Windows device name, and in a name with a folder no part may start with .. / is the only separator. Every accepted name is also one that snapshotPathOk (cli/pull_file.zig) accepts for a hub path, under stricter rules; sub/../outside, /abs and .cache/x stay refused. Reads follow a linked folder as they followed a linked top-level shard on 1.1.2 (a hub cache's layout).

The writes keep plain names because the link checks before --slide do not look at a shard's folder. shard_edit.linked reads only the model folder's own entries, so it never sees a shard in a folder. Flash Next's and Qwen3.5's checks lstat each index path, which follows a linked folder to the file inside it. So a linked folder would pass all three checks unseen.

With the index check relaxed and no startup refusal, --slide would load such a checkpoint and meet shard_edit's refusal only at its first save, after save_guard.begin has marked the save pending, so the folder would not open again until tensorfold slide rollback. slide_adapter.refusal, which both the CUDA and the Metal startup call, now refuses it after the link checks, with the shard's name. That includes nvidia/Qwen3.8-Flash-Next-FP4-Local, which 1.1.2 cannot load at all: with this change it serves again, without --slide. Learning into shards in folders could come later, with the link checks walking every part of a shard's path; this change keeps that choice open.

Draft until the GPU receipt is in: FP4-Local loading on 1.1.2 with this branch and #1050, which we run next on two GB10s.

Receipt

  • Environment: this branch (098ab795) on 1.1.2 (4ff78441), Zig 0.17.0; host tests on x86-64 Linux. GPU runs: to follow.
  • Exactness: n/a for this change, which checks the index before any tensor is read. The GPU receipt will show FP4-Local's ids at TP=2 on 1.1.2 with this branch and glm/cuda: load and serve nvidia/GLM-5.3-Flash-FP4-Local as stored #1050, against glm/cuda: load and serve nvidia/GLM-5.3-Flash-FP4-Local as stored #1050's ids on 1.1.1.
  • Decode speed before and after: n/a, the index is checked once at load.
  • Prompt speed before and after: n/a, as above.
  • Tests run, on x86-64 Linux from git archive trees of this branch and of 1.1.2:
    • The no-GPU workflow's steps. zig build test passes 3,451 of 3,467 with 15 skipped and 1 failing, cuda_cost_identity's "stable reads reuse a drafter, while a changed payload at the same path invalidates it", which fails the same way on 1.1.2 (3,443 of 3,459 pass there). test-kernel-contract, test-golden, test-server-security test-pull-security, fuzz-cortheon and zig build pass, and the tools/zig unittest (flashnext_batch_serve_test.py) passes 8 of 8. zig fmt --check passes on the seven files. tools/zig/lean_check.py: 438 problems on this branch and 438 on 1.1.2, none new.
    • The new tests' own steps, each passing on this branch and on 1.1.2: test-checkpoint-security (47 of 47 here), test-slide-adapter-host (2 of 2 here; it is not part of zig build test, so CI does not run the startup refusal test) and test-bolt-host, which runs RADIO's tests.
  • Not run, and why: Metal and Windows, for want of a Mac and a Windows host. On macOS, zig build test runs the new security, index and RADIO tests, and test-slide-adapter-host runs the startup refusal, which Metal's --slide startup also calls (native/metal.zig). On Windows the linked-folder test skips, as the existing symlink test does.

Checklist

  • Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off. (The change checks an index at load; rounds are unchanged.)
  • Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (No inference path changes: the index is checked before loading.)
  • No precision traded for speed. If the bits change, the description says which sums change and why. (No sum or kernel changes.)
  • A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt. (No kernel changes.)
  • Prompt processing is no slower than the last release. (By reasoning: one index check at load, no round or kernel changes.)
  • The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not. (With the GPU receipt.)
  • New tests fail before the change, pass after it, and skip cleanly without their dependency. (On 1.1.2 with these security tests, the folder test fails, 45 of 46 in test-checkpoint-security. With shard_edit put back on weightMap, both Living Weights tests fail, 45 of 47. With RADIO's basename comparison put back, its new test fails in test-bolt-host. Without the new line in refusal, the startup refusal test fails, 1 of 2. The index's part-rule test calls a function this change adds. The linked-folder test skips on Windows, as the existing symlink test does.)
  • Comments and docstrings are one line. No measurements or history in the source.
  • No personal data, machine names, internal hosts or local paths. No AI attribution lines.

…older; shard edits keep plain names

1.1.2's shared index check (46ce6df) takes a shard only as a plain file name, so an index that maps tensors into a
folder of the model folder is refused with BadIndex. nvidia/GLM-5.3-Flash-FP4-Local (ashhart#1050) is stored that way: its
index at revision 76348ddb maps 214 of its 295,041 tensors to tensors/native-QB.safetensors, the layout ashhart#1022's
cuda_memory test sizes.

- weightMap takes a plain name, or plain names joined by '/' with none starting with '.': each part goes through
  plainName (no empty part, no '.' or '..', no drive, backslash or device name), as cli/pull_file.zig's
  snapshotPathOk checks a hub path, so a shard stays inside the model folder and out of its hidden folders.
- shard_edit's bake, grow and overwrite, through which Living Weights, --slide and LoRA merges write shards, check
  with plainWeightMap and keep plain names: the link checks before --slide look at a shard's own entry only
  (shard_edit.linked at the folder's entries, Flash Next's and Qwen3.5's at each index path's last part), so a shard
  in a folder, or a linked folder, would pass them unseen.
- --slide refuses such a checkpoint at startup, after its link checks (slide_adapter.refusal), rather than at its
  first save, where shard_edit's refusal would leave the save pending until a rollback.
- RADIO's image shards compare the index's names with the shard paths past the model folder, not their basenames.
- Tests: the part rules in checkpoint_index.zig. In the security tests, a checkpoint with its shard in a folder
  opens; overwrite, grow and bake refuse such an index before writing; an overwrite through a linked folder leaves
  the file outside unchanged; sub/model.safetensors leaves the list of names that escape the folder. The slide
  adapter's tests hold the startup refusal; RADIO finds its image shard in a folder.
ashhart added a commit that referenced this pull request Oct 22, 2026
… folder's shard

Its fixture expected an index that names part/one.safetensors to be refused, which #1179 now allows. It
now writes a shard in a folder and checks the exact byte total, and keeps refusals for names that leave
the model folder (../, part/../ and an absolute path) beside the existing missing-file case.
@ashhart

ashhart commented Oct 22, 2026

Copy link
Copy Markdown
Owner

Thanks @Bizuayeu, we carried this onto main ourselves so the fix for #1187 doesn't wait: d3707df with you as the author, and one small commit of ours on top (7a90efa) that updates the Flash Next weight-stats test, which still expected a folder name to be refused; it now reads a shard in a folder and keeps refusing ../, part/../ and absolute names. We kept your rule as it is: every part of a shard name is checked, writers (grow, overwrite, bake, LoRA merges) still take plain names only, and --slide refuses a folder index at startup. A symlinked folder inside the model folder is followed like a symlinked shard already was, which Hugging Face's cache layout relies on; the index itself still cannot leave the folder. The host suites pass on main with both commits, and it ships in the next release.

- Synthema, Ash's AI Assistant.

@Bizuayeu

Copy link
Copy Markdown
Contributor Author

we carried this onto main ourselves so the fix for #1187 doesn't wait: d3707df with you as the author, and one small commit of ours on top (7a90efa) that updates the Flash Next weight-stats test, which still expected a folder name to be refused

#1179 (comment)

Thank you for carrying it, and for 7a90efa. Our checks ran the no-GPU workflow's steps, but not test-flashnext-buffer-plan-host, where that test runs; for changes like this we now also run the host steps whose tests use what the change touches.

ashhart pushed a commit that referenced this pull request Oct 24, 2026
nvidia/Qwen3.8-Flash-Next-FP4-Local maps every tensor to
tensors/model-000NN-of-00021.safetensors. Core's index check accepts that
since #1179, but the family's own admission refused any '/' with
UnsafeFlashShard, so serve stopped before CUDA init.

- families/flashnext/index.zig: shardName/plainPart take the same rule as
  core/checkpoint_index.zig, kept local so this file still imports no core;
  escapes (../, absolute, a drive, a hidden or reserved part) stay refused.
  Tests: the 30-shard fixture admitted flat, under tensors/, and refused
  under ../.
- core/checkpoint.zig: a checkpoint whose index names shards in a folder
  opens and reads them (shardFiles + openModel + get).
- cli/pull_test.zig: the fake hub serves tensors/model-…safetensors; the
  main pull test asserts the snapshot's subfolder link, its blob and the
  second pull's "cached", and a security test pins the nested path.
- cli/hub.zig: sizeOf counts shards in a subfolder (the info memory floor);
  the registry test no longer hardcodes a Metal-only family.
- build: test-pull-host runs pull's tests with the engine registry; the
  older test-pull-security step lacks native_engines and does not compile.

Evidence: zig build test-pull-host (21 tests) and zig build test-cuda-host
(176/176 steps) pass on Linux x86_64 with Zig 0.17.0; zig fmt --check clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants