Skip to content

[None][feat] RI-01 pin shared KV contracts and staging compatibility - #19823

Draft
chienchunhung wants to merge 2 commits into
NVIDIA:mainfrom
chienchunhung:dev/ri-01-shared-kv-contracts
Draft

chienchunhung wants to merge 2 commits into
NVIDIA:mainfrom
chienchunhung:dev/ri-01-shared-kv-contracts

Conversation

@chienchunhung

@chienchunhung chienchunhung commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Establish the content-addressed KV backend contract and its mapping to manager-owned staging. Shared consumers can distinguish misses and partial delivery from failure, while logical outcomes remain separate from proof that backend memory access has ended.

Scope

  • Add the 13-symbol shared backend surface, retaining the existing paired API until its later convergence.
  • Add profile validation and staging conversion: preserve lender-generated opaque names, validate local slot geometry, and translate served sets into whole-row masks.
  • Pin the contract revision and require explicit backend/layout/topology facts for the first BF16 MHA / TRTLLM HND / TP=DP=PP=CP=1 profile. These checks do not enable or qualify a runtime path.
  • Document content versus physical identity and readiness obligations. No new allocator, lifecycle state machine, scheduler integration, or third-party dependency.

Verification

At 8562acc19139f40b6cb889d9bcf26f52e9333195:

  • Focused CPU tests: 86 passed across test_shared_backend_contract.py, test_shared_resource_contract.py, and the prerequisite lender's test_names.py.
  • Executed on macOS/Python 3.12 with NumPy; an isolated package-path harness imports the real production modules while excluding CUDA package initialization and repository GPU fixtures. Normal repository collection remains for CI.
  • Installed commit hooks and git diff --check: passed.
  • Independent verification at the final head: 86 passed, 0 skipped from an immutable source archive. Complete self-review after the final clarification found no actionable issues.
  • Full CI launched for this head: PR_Github #76142, with bot acknowledgment. The pipeline is pending; no passing result is claimed.
  • RI-01 changes pure contracts/mapping only; real-lender GPU composition is part of RI-02, and deployment qualification remains separate.

Notes

RI-01 is stacked on open prerequisite #19722. This draft against main includes its separate commit; RI-01-only change is six files. Rebase after the lender lands. The later RI-02 binding also consumes #19822.

  • Reviewed the repository PR checklist for this scope; tests and developer documentation included. No protected user-facing API or dependency changes.

Add request-scoped staging and in-place lenders for KVCacheManagerV2, with opaque content identities, layout derivation, registration holds, and lease lifecycle handling.

Include manager lifecycle and shrink hooks, accumulated fetch readiness, and layout, public API, staging, in-place, and host-tier tests.

Signed-off-by: Shixiaowei02 <[email protected]>
@chienchunhung
chienchunhung force-pushed the dev/ri-01-shared-kv-contracts branch from 9c8df75 to 8562acc Compare October 2, 2026 23:39

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #76142 [ run ] triggered by Bot. Commit: 8562acc Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #76142 [ run ] completed with state FAILURE. Commit: 8562acc
/LLM/main/L0_MergeRequest_PR pipeline #62773 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants