Repository navigation
flashnext: a shard in a folder inside the model folder (#1220) - #1221
paragontasx wants to merge 2 commits into
Conversation
nvidia/Qwen3.8-Flash-Next-FP4-Local maps every tensor to tensors/model-000NN-of-00021.safetensors. Core's index check accepts that since ashhart#1179, but the family's own admission refused any '/' with UnsafeFlashShard, so serve stopped before CUDA init. - families/flashnext/index.zig: shardName/plainPart take the same rule as core/checkpoint_index.zig, kept local so this file still imports no core; escapes (../, absolute, a drive, a hidden or reserved part) stay refused. Tests: the 30-shard fixture admitted flat, under tensors/, and refused under ../. - core/checkpoint.zig: a checkpoint whose index names shards in a folder opens and reads them (shardFiles + openModel + get). - cli/pull_test.zig: the fake hub serves tensors/model-…safetensors; the main pull test asserts the snapshot's subfolder link, its blob and the second pull's "cached", and a security test pins the nested path. - cli/hub.zig: sizeOf counts shards in a subfolder (the info memory floor); the registry test no longer hardcodes a Metal-only family. - build: test-pull-host runs pull's tests with the engine registry; the older test-pull-security step lacks native_engines and does not compile. Evidence: zig build test-pull-host (21 tests) and zig build test-cuda-host (176/176 steps) pass on Linux x86_64 with Zig 0.17.0; zig fmt --check clean.
Served receipt (one GB10, this branch)Environment: NVIDIA GB10 (DGX Spark), aarch64, driver 580.173.02, CUDA 13.0, nvcc 13.0.88, Zig 0.17.0, Startup, unedited: The publisher's Requests (greedy,
So, on this box and this branch: drafted == Not run here: vision (server started without |
`pull_test.zig` ran in no passing no-GPU step: `test-pull-security` had not compiled since `cli/hub.zig` began reading `native_engines` for `family()`, and `cli/tests.zig` is built with two name filters only. - new `zig/build/pull_tests.zig` owns the pull test module — the engine registry, EXLR8's import, the `test-pull-host` step and its dependency on `test-cuda-host`; it returns early on Windows, where the fake hub's POSIX socket (`sockaddr.in`) cannot compile. - `golden.step` takes the platform's engine registry, so `test-pull-security` compiles again: the CUDA host passes `native_modules.engines`, the Metal host its own `engines`. - `cuda.zig` stays within the 600-line lean limit; the call replaces a blank line beside the other Windows-guarded host test. Verified: `zig build test-pull-host test-pull-security` passes; lean check reports 0 new problems against 8e49cf8; `zig fmt --check` clean.
0d6c7b0 to
9ec3f52
Compare
|
Thanks @paragontasx. This is on - Synthema, Ash's AI Assistant. |
|
This shipped in TensorFold 1.1.4: https://github.com/ashhart/TensorFold/releases/tag/v1.1.4 Thank you @paragontasx for the contribution. - Synthema, Ash's AI Assistant. |
What this changes
nvidia/Qwen3.8-Flash-Next-FP4-Localmaps every tensor totensors/model-000NN-of-00021.safetensors.#1179 lets the core index accept a shard in a folder inside the model folder, but Flash Next validates the weight map
a second time before any header is read, and that gate refused any
/withUnsafeFlashShard— soservestoppedbefore CUDA init. This is the family half of #1187; filed as #1220.
families/flashnext/index.zig:shardName/plainPartnow take the same rule ascore/checkpoint_index.zig(plain names joined by
/, no empty, hidden, dotted-tail or reserved part, and the file must be.safetensors),so
tensors/model-…safetensorsis admitted while../, absolute paths, drive names and device stems stay refused.It is kept local because this file imports no core by design. Tests: the committed 30-shard fixture is admitted
flat, admitted identically under
tensors/, and refused under../; plus an accept/refuse table.core/checkpoint.zig: a checkpoint whose index names shards in a folder opens and reads a tensor(
shardFiles+openModel+get), so the loader itself is covered, not only the index.cli/pull_test.zig: the fake hub servestensors/model-…safetensors; the main pull test asserts the snapshot'ssubfolder link resolves to the verified blob and that a second pull reports
cached; asecurity:test pins thatthe nested path stays inside the snapshot. The new sha hex is freed in
stopServer.cli/hub.zig:sizeOfcounts shards in a subfolder (this is theinfomemory floor a nested checkpoint wouldotherwise undercount); the registry test no longer hardcodes
gemma4_unified, which only the Metal registry serves— it panicked when the module was tested with the CUDA registry.
pull_test.zigpreviously ran in no passing no-GPU step.zig/build/pull_tests.zigaddstest-pull-host(the whole pull suite, part oftest-cuda-host, skipped on Windows where the fake hub’s POSIX socket cannot compile), andgolden.stepnow takes the platform's engineregistry so
zig build test-pull-securitycompiles again — it had been failing the no-GPU CI job sincehub.zigbegan reading
native_engines. The CUDA host passesnative_modules.engines, the Metal host its ownengines.No kernel changes.
Receipt
Environment: Linux x86_64, Zig 0.17.0, no GPU;
main@8e49cf8plus this branch.zig build test-pull-host→ 21 tests passzig build test-cuda-host --summary all→ 176/176 steps succeededzig test zig/src/families/flashnext/index.zig→ 2/2zig fmt --check→ cleanNot run, and why
The GPU path: this machine has no NVIDIA device, and the checkpoint-level proof here is the publisher's own index
layout (30 shards, 3,747 tensors) rather than a load. I have a GB10 available and will send a served receipt
(
serve nvidia/Qwen3.8-Flash-Next-FP4-Local,--prompt-cache-gib 0) against this branch next, unless you wouldrather take it from here.
Fixes #1220. Remaining half of #1187.