Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ This entry describes the release candidate; it does not authorize or attest publ
- Hardened deterministic backend configuration, binding/export/package checks, package curation, and multi-architecture Triton packaging: all six vendored AOT trees are embedded, exact-SM dispatched, and paired with portable fallback on other accepted targets.
- Removed arbitrary package size and file-count caps for both crates. File counts, compressed sizes, regular-file payload sums, and SHA-256 hashes remain diagnostics, not release limits; archive inventories must match Cargo's package lists, and content, provenance, extracted-build, and downstream compatibility checks remain required.
- Contained the entire Rust logits-processor trampoline lifetime, including final capture destruction after concurrent registration teardown, while preserving callback panic reporting and panic-payload disposal.
- Pinned online GGUF metadata and body retrieval to a validated immutable commit; explicit full-SHA requests reject different metadata or offline snapshots. Validate downloads before explicitly updating requested refs, and repair hf-hub's implicit pinned-SHA ref writes on rejection without rolling back mutable aliases. Offline resolution still requires a cache ref; mutable `main` remains the default requested revision.
- Pinned online GGUF and Safetensors metadata and body retrieval to a validated immutable commit; explicit full-SHA requests reject different metadata or offline snapshots. Validate downloads before explicitly updating requested refs, and repair hf-hub's implicit pinned-SHA ref writes on rejection without rolling back mutable aliases. Offline resolution still requires a cache ref; mutable `main` remains the default requested revision.
- Native C API fixture failures now retain the default scheduler's failing status; an alternate scheduler run cannot make the gate pass. Callback counts are not token counts, so the upstream callback-count assertion alone does not establish token loss.
- Optional ASan/UBSan and native-only TSan recipes select exact package/target test executables from Cargo JSON, reject empty suites, and require each exact TSan filter to pass one named test. Removed the stale release number from the CUDA/Vulkan conflict diagnostic.
- Simplified duplicated packaging policy and release documentation, and replaced the video downloader/mux application with a minimal local-checkpoint PPM/WAV example. Public library APIs and centralized native-buffer safety checks are unchanged.
- Simplified duplicated packaging policy and release documentation, and replaced the video downloader/mux application with a minimal local-checkpoint PPM/WAV example. Narrowed the committed WAV fixture reader, repeated link-mode tests, native-gate status plumbing, and redundant build/sampling checks. Bundled builds now watch the complete native include directory. Public library APIs and centralized native-buffer safety checks are unchanged.

### Added

Expand Down
21 changes: 6 additions & 15 deletions Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ _link-modes crate_root target_base:
bundled_static_target="$work/bundled-static"
echo '==> bundled static'
CARGO_TARGET_DIR="$bundled_static_target" \
cargo test --locked -p vllm-cpp-sys --release --tests
cargo test --locked -p vllm-cpp-sys --release --test layout --test symbols

bundled_dynamic_target="$work/bundled-dynamic"
echo '==> bundled dynamic build'
Expand All @@ -221,7 +221,7 @@ _link-modes crate_root target_base:
echo '==> bundled dynamic test'
LD_LIBRARY_PATH="$bundled_dynamic_lib_dir${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" \
CARGO_TARGET_DIR="$bundled_dynamic_target" \
cargo test --locked -p vllm-cpp-sys --release --tests --features dynamic-link
cargo test --locked -p vllm-cpp-sys --release --test layout --test symbols --features dynamic-link

prefix="$work/system-prefix"
mkdir -p "$prefix/include" "$prefix/lib" "$prefix/lib64" "$prefix/blake3-lib"
Expand All @@ -242,7 +242,7 @@ _link-modes crate_root target_base:
VLLM_CPP_ROOT="$prefix" \
VLLM_CPP_BLAKE3_LIB_DIR="$prefix/blake3-lib" \
CARGO_TARGET_DIR="$system_static_target" \
cargo test --locked -vv -p vllm-cpp-sys --release --tests \
cargo test --locked -vv -p vllm-cpp-sys --release --test layout --test symbols \
--no-default-features --features system 2>&1 | tee "$system_static_log"
grep -Fq "cargo:rerun-if-changed=$prefix/lib/libvllm.a" "$system_static_log"
grep -Fq \
Expand All @@ -269,7 +269,7 @@ _link-modes crate_root target_base:
VLLM_CPP_ROOT="$prefix" \
VLLM_CPP_BLAKE3_LIB_DIR="$prefix/blake3-lib" \
CARGO_TARGET_DIR="$system_static_target" \
cargo test --locked -vv -p vllm-cpp-sys --release --tests \
cargo build --locked -vv -p vllm-cpp-sys --release \
--no-default-features --features system 2>&1 | tee "$system_static_rerun_log"
grep -Fq 'Dirty vllm-cpp-sys' "$system_static_rerun_log"
grep -Fq 'libvllm.a' "$system_static_rerun_log"
Expand Down Expand Up @@ -317,7 +317,7 @@ _link-modes crate_root target_base:
LD_LIBRARY_PATH="$prefix/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" \
VLLM_CPP_ROOT="$prefix" \
CARGO_TARGET_DIR="$system_dynamic_target" \
cargo test --locked -vv -p vllm-cpp-sys --release --tests \
cargo test --locked -vv -p vllm-cpp-sys --release --test layout --test symbols \
--no-default-features --features system,dynamic-link 2>&1 \
| tee "$system_dynamic_log"
grep -Fq "cargo:rerun-if-changed=$prefix/lib/libvllm.so" \
Expand Down Expand Up @@ -723,19 +723,10 @@ native-capi:
[[ $test_count -eq 1 ]]
grep -Fxq 'Total Tests: 1' "$listing"

set +e
env -u VT_ASYNC_SCHED -u VT_ASYNC_RUNNER \
ctest --test-dir "$build" --output-on-failure --tests-regex '^test_capi$' \
2>&1 | tee "$output"
default_status=${PIPESTATUS[0]}
set -e
if [[ $default_status -eq 0 ]]; then
grep -Fxq '100% tests passed, 0 tests failed out of 1' "$output"
exit 0
fi

echo "default native C API gate failed (status $default_status); alternate scheduler runs cannot satisfy this gate" >&2
exit "$default_status"
grep -Fxq '100% tests passed, 0 tests failed out of 1' "$output"

# Run sys-first crates.io publication checks without uploading.
publish-dry-run:
Expand Down
2 changes: 1 addition & 1 deletion vllm-cpp-sys/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ fn main() {
"src/build_support.rs",
"vllm.cpp/CMakeLists.txt",
"vllm.cpp/cmake",
"vllm.cpp/include/vllm.h",
"vllm.cpp/include",
"vllm.cpp/src",
"vllm.cpp/tests/vt/test_rocm_backend.cpp",
"vllm.cpp/triton_kernels",
Expand Down
12 changes: 0 additions & 12 deletions vllm-cpp-sys/src/build_config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -341,12 +341,6 @@ fn validate_cutlass(
root.display()
))
})?;
if !canonical.is_absolute() {
return Err(ConfigError::new(format!(
"VLLM_CPP_CUTLASS_DIR must resolve to an absolute path; got {}",
canonical.display()
)));
}

for relative in ["include/cutlass/cutlass.h", "include/cutlass/version.h"] {
let path = canonical.join(relative);
Expand Down Expand Up @@ -423,12 +417,6 @@ fn validate_mlx(inputs: &Inputs, probe: &impl PathProbe) -> Result<Option<PathBu
root.display()
))
})?;
if !canonical.is_absolute() {
return Err(ConfigError::new(format!(
"MLX_ROOT must resolve to an absolute path; got {}",
canonical.display()
)));
}

for relative in [
"include/mlx/array.h",
Expand Down
6 changes: 0 additions & 6 deletions vllm-cpp-sys/tests/build_config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -267,12 +267,6 @@ fn mlx_validation_reports_each_configuration_error() {
format!("{ERROR_PREFIX} MLX_ROOT=mlx is not an existing directory")
);

let mut relative_probe = MockProbe::mlx();
relative_probe
.canonical
.insert(PathBuf::from("mlx"), PathBuf::from("canonical-mlx"));
assert!(error(&inputs, &relative_probe).contains("must resolve to an absolute path"));

for missing in [
"include/mlx/array.h",
"lib/libmlx.dylib",
Expand Down
2 changes: 2 additions & 0 deletions vllm-cpp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ let engine = Engine::load(path)?;

GGUF mode retrieves one root-level lowercase `.gguf` file and rejects split sets. Safetensors mode first reads repository metadata for `main` or the explicit revision, pins downloads to its commit SHA, and retrieves only the root files required by this native loader: `config.json`, `tokenizer.json`, optional `tokenizer_config.json`, and either `model.safetensors` or the root index plus every indexed root shard. It returns the shared `snapshots/<sha>` directory and creates that revision's cache ref only after a complete successful retrieval. It does not download unrelated repository assets. Offline mode constructs no network API, reads the cached `main` ref by default, and distinguishes a missing cache revision from an incomplete cached snapshot.

Both formats require a full commit SHA in online metadata and reject metadata or offline snapshots that differ from an explicit full-SHA request. Offline resolution still requires the requested revision's cache ref, even for a full SHA.

Retrieval validates cache and snapshot completeness; it does not prove that the pinned native engine supports the repository's model architecture, tokenizer, quantization, or backend.

In the repository checkout, `just setup-test-model` explicitly resolves `Qwen/Qwen3-0.6B` at immutable revision `c1899de289a04d12100db370d81485cdf75e47ca`, reusing the standard Hugging Face cache and honoring normal `HF_HOME` and authentication. Ordinary tests and instrumentation commands never resolve or download this fixture. `VLLM_CPP_TEST_MODEL` remains an external contract: when supplied, it must point to a prepared model directory; model tests skip when it is unset.
Expand Down
2 changes: 1 addition & 1 deletion vllm-cpp/examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ The bare path remains an alias for `local`. Hub forms enable download progress,

Omitting `--revision` follows the Hub's mutable default `main` branch. Use `.revision(...)` in the library or `--revision` in an example to select a branch, tag, or commit; an immutable commit SHA is recommended for reproducibility. The resolver is synchronous and always available because `hf-hub` is a normal dependency, not a Cargo feature.

By default the resolver uses Hugging Face's normal cache, honoring `HF_HOME`, and selects the cached token created by Hugging Face login. Library callers can use `.revision(...)`, `.cache_dir(...)`, `.token(...)`, `.progress(...)`, and `.offline(true)`. An explicit token overrides the selected cached token and is redacted from resolver `Debug`. Offline default resolution reads only the cached `main` ref; an explicit revision reads only that cached ref. Offline mode constructs no API. The official Hugging Face endpoint remains fixed; `HF_ENDPOINT` is not used.
By default the resolver uses Hugging Face's normal cache, honoring `HF_HOME`, and selects the cached token created by Hugging Face login. Library callers can use `.revision(...)`, `.cache_dir(...)`, `.token(...)`, `.progress(...)`, and `.offline(true)`. An explicit token overrides the selected cached token and is redacted from resolver `Debug`. Offline default resolution reads only the cached `main` ref; an explicit revision reads only that cached ref. Both formats reject online metadata or cached snapshots that differ from an explicit full-SHA request; online metadata must contain a full commit SHA. Even full-SHA offline requests require a cache ref. Offline mode constructs no API. The official Hugging Face endpoint remains fixed; `HF_ENDPOINT` is not used.

GGUF mode retrieves one safe root-level filename ending in lowercase `.gguf`; split GGUF sets are unsupported. Safetensors mode queries metadata for `main` or the explicit revision, pins file retrieval to the returned commit SHA, and creates that revision's cache ref after complete success. It retrieves only `config.json`, `tokenizer.json`, optional `tokenizer_config.json`, and either root `model.safetensors` or root `model.safetensors.index.json` plus every unique root shard in `weight_map`. Unrelated repository assets are not downloaded. Successful retrieval establishes a complete sparse snapshot for the pinned loader, not model-family or backend compatibility.

Expand Down
Loading
Loading