Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,26 @@ All notable changes to this project will be documented in this file.

## [0.0.2] - 2026-08-27

This entry describes the release candidate; it does not authorize or attest publication.

### Changed

- Pinned bundled native vllm.cpp to tag `v0.0.2`, commit `7020de93652ca920424a10ac5255b34810dd2f24`, moving the stable C contract from ABI 10 with 19 functions to ABI 17 with 35 functions; system libraries must match the new ABI, layouts, signatures, and exports.
- Model construction now obtains native defaults and overlays only explicit Rust settings, preserving helper defaults including `block_size=32`, `max_num_seqs=32`, and `gpu_memory_utilization=0.92`.
- Hardened deterministic backend configuration, binding/export/package checks, package curation, and multi-architecture Triton packaging: all six vendored AOT trees are embedded, exact-SM dispatched, and paired with portable fallback on other accepted targets.
- Raised the safe crate unpacked package limit to 512 KiB while retaining its 40-file and 128-KiB compressed limits; sys limits remain 1,400 files, 40 MiB unpacked, and 6 MiB compressed.
- Removed arbitrary package size and file-count caps for both crates. File counts, compressed sizes, regular-file payload sums, and SHA-256 hashes remain diagnostics, not release limits; exact inventories and all content, provenance, extracted-build, and downstream compatibility checks remain required.
- Contained the entire Rust logits-processor trampoline lifetime, including final capture destruction after concurrent registration teardown, while preserving callback panic reporting and panic-payload disposal.
- Pinned online GGUF metadata and body retrieval to a validated immutable commit; explicit full-SHA requests reject different metadata or offline snapshots. Validate downloads before explicitly updating requested refs, and repair hf-hub's implicit pinned-SHA ref writes on rejection without rolling back mutable aliases. Offline resolution still requires a cache ref; mutable `main` remains the default requested revision.
- Native C API fixture failures now retain the default scheduler's failing status; an alternate scheduler run cannot make the gate pass. Callback counts are not token counts, so the upstream callback-count assertion alone does not establish token loss.
- Optional ASan/UBSan and native-only TSan recipes select exact package/target test executables from Cargo JSON, reject empty suites, and require each exact TSan filter to pass one named test. Removed the stale release number from the CUDA/Vulkan conflict diagnostic.

### Added

- `Device::{Auto, Cpu, Cuda}` selection and KV-memory controls with native precedence `num_blocks > kv_cache_memory_bytes > gpu_memory_utilization` and profile/fallback sizing.
- Pre-tokenized `Engine::complete_tokens` with owned token output, optional copied completion metadata, and explicit truncation reporting.
- Thread-local, exclusive `TranscriptionEngine` and `EmbeddingEngine` owners with borrowed inputs and Rust-owned transcription and row-major embedding results.
- Separate `VideoEngine` checkpoint-set ownership, blocking generation parameters/results, and standalone `VideoMuxParams`/`VideoMuxArgv` composition that never executes ffmpeg.
- Model-free acceptance gates for docs, tests, ABI/layout/signature/exports, native fixtures, sanitizers, link modes, packages, downstream consumers, publish dry-run, and exact MSRV.
- Model-free acceptance gates for docs, tests, ABI/layout/signature/exports, native fixtures, sanitizers, link modes, packages, downstream consumers, publish dry-run, and exact MSRV. Focused Rust and tooling regressions cover callback destruction, loopback Hub branch movement/cache identity, default native-gate status, mixed artifacts, missing filters, and filesystem-independent payload accounting. These gate definitions are not evidence of an exact-candidate pass.

### Compatibility

Expand Down
296 changes: 109 additions & 187 deletions Justfile

Large diffs are not rendered by default.

8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ just sanitizers "$model"
just tsan "$model"
```

These commands are optional evidence and are recorded only when rerun against the exact candidate. The TSan recipe instruments native C++ only and does not claim Rust standard-library race coverage. `VLLM_CPP_SANITIZE` is a bundled-build test input; system mode rejects it.
These commands are optional evidence and are recorded only when rerun against the exact candidate. Sanitizer recipes use exact package/target executables from Cargo JSON, require nonzero complete suites, and reject an exact TSan filter unless one named test passes. The TSan recipe instruments native C++ only and does not claim Rust standard-library race coverage. `VLLM_CPP_SANITIZE` is a bundled-build test input; system mode rejects it.

## Link Modes

Expand All @@ -131,13 +131,15 @@ cargo package -p vllm-cpp --locked --list
just package-test
```

The package gate validates deterministic inventories for both crates, package metadata, required source/docs/examples/tests/licenses/native inputs, forbidden payloads, and license provenance. It extracts and tests both crates offline, then runs independent sys and safe downstream consumers; the safe consumer resolves both extracted crates rather than this workspace. The sys package carries only native build inputs and required licenses/notices; upstream tests, large fixtures, media, benchmarks, fetched SDKs, external CUTLASS trees, and agent records are excluded.
The package gate reports file counts, compressed sizes, payload bytes, and SHA-256 hashes as diagnostics only; neither crate has an arbitrary numeric package size or file-count cap. Payload bytes are the sum of regular archive member sizes, not filesystem-dependent extracted directory sizes. It validates deterministic inventories for both crates, package metadata, required source/docs/examples/tests/licenses/native inputs, forbidden payloads, and license provenance. It extracts and tests both crates offline, then runs independent sys and safe downstream consumers; the safe consumer resolves both extracted crates rather than this workspace. The sys package carries only native build inputs and required licenses/notices; upstream tests, large fixtures, media, benchmarks, fetched SDKs, external CUTLASS trees, and agent records are excluded.

`just publish-dry-run` performs a sys-then-safe workspace packaging dry-run without uploading; it uses `--no-verify` to avoid the pre-publication registry cycle. As required by [RELEASING.md](RELEASING.md), after `vllm-cpp-sys` is available from crates.io, run the full `cargo publish -p vllm-cpp --locked --dry-run` verification before publishing the safe crate.

## Validation Boundary

Mandatory candidate evidence is Linux x86_64 CPU and model-free: formatting, lint, docs, workspace tests, generated-binding/header/layout/signature/ABI/exact-export checks, all four CPU link modes, native C API fixtures, ASan/UBSan/leak checks over committed fixtures, package extraction/downstream checks, publication dry-run, and exact MSRV validation.
Mandatory candidate evidence is Linux x86_64 CPU and model-free: formatting, release-tooling regressions, lint, docs, workspace tests, generated-binding/header/layout/signature/ABI/exact-export checks, all four CPU link modes, native C API fixtures, ASan/UBSan/leak checks over committed fixtures, package extraction/downstream checks, publication dry-run, and exact MSRV validation.

`just native-capi` preserves the default scheduler result with `VT_ASYNC_SCHED` and `VT_ASYNC_RUNNER` unset. A failing default fixture blocks the candidate; a separate synchronous diagnostic run cannot make it green. Callback cardinality is not token cardinality. Dirty-tree development passes are not a substitute for independently approved clean-commit validation in [RELEASING.md](RELEASING.md).

Prepared Qwen inference/sanitizers, native-only TSan, successful Rust MiniMax-H3 generation, Miri, Linux ARM64, Apple ARM64, Vulkan, CUDA/CUTLASS/Triton, Metal/MLX, and accelerator runtime are optional or deferred lanes. The manual platform workflows are configuration, not exact-candidate evidence unless separately dispatched and recorded.

Expand Down
12 changes: 8 additions & 4 deletions RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Releases are prepared and published manually. A candidate pass or dry-run does not authorize a Git tag, crates.io upload, merge, push, or GitHub release. Publication is an irreversible registry action and requires separate maintainer authorization.

Uncommitted development runs may exercise the gates below, but must be reported as dirty-tree validation, not approved release proof. Formal release validation and publication require the independently approved exact commit and clean identities below.

## Prepare a candidate

1. Obtain `VLLM_CPP_RELEASE_COMMIT` from the independent approval of the exact release commit. Never derive the expected value from the current checkout. Require a lowercase, full 40-character SHA, verify that it names a commit object, and compare it with the root checkout before any candidate gate:
Expand Down Expand Up @@ -63,11 +65,13 @@ RUSTDOCFLAGS='-D warnings' cargo doc --locked --workspace --no-deps --features v
git diff --check
```

`just ci` includes formatting, warnings-denied lint/docs, model-free workspace tests, generated bindings and ABI conformance, four CPU link modes, the native C API fixture, model-free ASan/UBSan/leak checks, package/extracted/downstream validation, and no-upload publish dry-run. Run the exact MSRV gate separately so stable-toolchain success cannot mask it.
`just ci` includes formatting, deterministic release-tooling regressions, warnings-denied lint/docs, model-free workspace tests, generated bindings and ABI conformance, four CPU link modes, the native C API fixture, model-free ASan/UBSan/leak checks, package/extracted/downstream validation, and no-upload publish dry-run. Run the exact MSRV gate separately so stable-toolchain success cannot mask it.

The native fixture gate always runs with `VT_ASYNC_SCHED` and `VT_ASYNC_RUNNER` unset and preserves any default-run failure. No assertion is waived or excluded. A separate `VT_ASYNC_SCHED=0` run may help diagnose callback coalescing, but cannot satisfy the default gate or establish token loss from callback cardinality alone. Record default and diagnostic commands/statuses independently; a default failure is a candidate blocker. If `just ci` stops, run the remaining recipes separately to expose other blockers, not to relabel the aggregate as passed.

Nix support remains an optional convenience, not a prerequisite. Maintainers who choose it may run `nix develop -c env -u VLLM_CPP_TEST_MODEL just ci` and `nix develop .#msrv -c just msrv`; `nix flake check --no-build` is an additional Nix-specific evaluation check, not an ordinary release gate.
Nix support remains an optional convenience, not a prerequisite. Maintainers who choose it may run `nix develop --no-write-lock-file -c env -u VLLM_CPP_TEST_MODEL just ci` and `nix develop .#msrv --no-write-lock-file -c just msrv`; `nix flake check --no-build` is an additional Nix-specific evaluation check, not an ordinary release gate.

Prepared-Qwen inference/sanitizers, native-only TSan, successful Rust MiniMax-H3 generation, Miri, Linux ARM64, Apple ARM64, Vulkan, CUDA/CUTLASS/Triton, Metal/MLX, and accelerator runtime are optional or deferred. Record one only when it ran against the exact candidate; configured workflows and older results are not candidate evidence.
Prepared-Qwen inference/sanitizers, native-only TSan, successful Rust MiniMax-H3 generation, Miri, Linux ARM64, Apple ARM64, Vulkan, CUDA/CUTLASS/Triton, Metal/MLX, and accelerator runtime are optional or deferred. Record one only when it ran against the exact candidate; configured workflows and older results are not candidate evidence. Sanitizer recipes select one package/target test artifact from Cargo JSON rather than target-directory timestamps. Full suites must report matching nonzero running/passed counts, and every exact TSan filter must report one named passed test; a missing filter is a failure, not coverage.

## Inspect packages

Expand All @@ -83,7 +87,7 @@ For both runs, retain:

- sorted `cargo package --list` inventories and archive basenames;
- normalized `Cargo.toml` manifests, including exact `vllm-cpp-sys =0.0.2`;
- file counts and unpacked/compressed sizes, enforcing sys limits of 1,400 files, 40 MiB unpacked, and 6 MiB compressed and safe limits of 40 files, 512 KiB unpacked, and 128 KiB compressed;
- file counts and payload/compressed sizes as diagnostics only, with no arbitrary numeric package size or file-count caps for either crate; payload means the sum of regular archive member sizes for both crates, excluding directory metadata and archive headers, not `du` of the extracted tree; exact inventory, content, provenance, and compatibility checks remain mandatory;
- complete license/notice/provenance inventories and checks that current READMEs, source, tests, examples, and required native/backend inputs are present;
- SHA-256 for both archives from each run and successful extracted/offline builds, all four extracted sys link modes, and independent sys and safe downstream consumers.

Expand Down
Loading
Loading