Skip to content

Add typed checkpoint weight records - #666

Draft
xiaoyu-work wants to merge 2 commits into
refactor/component-manifestfrom
refactor/typed-weight-pipeline
Draft

Add typed checkpoint weight records#666
xiaoyu-work wants to merge 2 commits into
refactor/component-manifestfrom
refactor/typed-weight-pipeline

Conversation

@xiaoyu-work

Copy link
Copy Markdown
Contributor

Summary

  • add typed WeightRecord, PackedWeight, and component-owned WeightBundle abstractions
  • group Olive and GPTQ/AWQ sidecars into one logical weight with completeness checks
  • add a codec registry keyed by quant_method
  • expose compatibility normalization through the existing proven packers
  • leave the active checkpoint loader unchanged in this foundational PR

Stack

Validation

  • 10 record/codec unit tests
  • 49 codec and existing Olive/GPTQ/AWQ compatibility tests

@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown

🏗️ Architecture Diff

Comparing 1f47daa15c0dfc

Model Sub-model Changes Status

No architecture changes detected.


Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed)

@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown

Performance Comparison

Comparing 1f47daa15c0dfc

Model Metric Baseline Current Delta
bert (feature-extraction) model_size_bytes 359 KB 359 KB +0.0%
bert (feature-extraction) num_nodes 68 68 +0.0%
falcon model_size_bytes 364 KB 364 KB +0.0%
falcon num_nodes 66 66 +0.0%
gemma2 model_size_bytes 428 KB 428 KB +0.0%
gemma2 num_nodes 105 105 +0.0%
gpt2 model_size_bytes 324 KB 324 KB +0.0%
gpt2 num_nodes 54 54 +0.0%
llama model_size_bytes 425 KB 425 KB +0.0%
llama num_nodes 60 60 +0.0%
llama (static-cache) model_size_bytes 425 KB 425 KB +0.0%
llama (static-cache) num_nodes 56 56 +0.0%
mamba (ssm-text-generation) model_size_bytes 296 KB 296 KB +0.0%
mamba (ssm-text-generation) num_nodes 94 94 +0.0%
phi3 model_size_bytes 421 KB 421 KB +0.0%
phi3 num_nodes 58 58 +0.0%
phi3 (static-cache) model_size_bytes 421 KB 421 KB +0.0%
phi3 (static-cache) num_nodes 54 54 +0.0%
qwen2 model_size_bytes 425 KB 425 KB +0.0%
qwen2 num_nodes 60 60 +0.0%
qwen2 (static-cache) model_size_bytes 425 KB 425 KB +0.0%
qwen2 (static-cache) num_nodes 56 56 +0.0%
qwen3_5_moe (hybrid-text-generation) model_size_bytes 506 KB 506 KB +0.0%
qwen3_5_moe (hybrid-text-generation) num_nodes 265 265 +0.0%
qwen3_5_text (hybrid-text-generation) model_size_bytes 458 KB 458 KB +0.0%
qwen3_5_text (hybrid-text-generation) num_nodes 127 127 +0.0%
qwen3_5_vl (hybrid-qwen-vl) model_size_bytes 977 KB 977 KB +0.0%
qwen3_5_vl (hybrid-qwen-vl) num_nodes 450 450 +0.0%
t5 (seq2seq) model_size_bytes 836 KB 836 KB +0.0%
t5 (seq2seq) num_nodes 176 176 +0.0%
whisper (speech-to-text) model_size_bytes 1008 KB 1008 KB +0.0%
whisper (speech-to-text) num_nodes 128 128 +0.0%

No performance regressions.

@xiaoyu-work
xiaoyu-work force-pushed the refactor/typed-weight-pipeline branch from b2b820e to fabb835 Compare August 27, 2026 02:21
@xiaoyu-work
xiaoyu-work force-pushed the refactor/typed-weight-pipeline branch from fabb835 to b1ea32f Compare August 27, 2026 02:41
@xiaoyu-work
xiaoyu-work force-pushed the refactor/typed-weight-pipeline branch from b1ea32f to e02875f Compare August 27, 2026 21:06
@xiaoyu-work
xiaoyu-work force-pushed the refactor/component-manifest branch from b549476 to 501c342 Compare August 27, 2026 21:13
@xiaoyu-work
xiaoyu-work force-pushed the refactor/typed-weight-pipeline branch from e02875f to ea9c144 Compare August 27, 2026 21:13
@xiaoyu-work
xiaoyu-work force-pushed the refactor/component-manifest branch from 501c342 to 1f47daa Compare August 27, 2026 21:30
Group affine packed sidecars into component-owned logical records and introduce a quantization codec registry over the existing Olive, GPTQ, and AWQ normalization helpers. The active loader remains unchanged.

Signed-off-by: Xiaoyu Zhang <[email protected]>
Update typed weight fixtures for the clarified ComponentDescriptor API.

Signed-off-by: Xiaoyu Zhang <[email protected]>
@xiaoyu-work
xiaoyu-work force-pushed the refactor/typed-weight-pipeline branch from ea9c144 to 15c0dfc Compare August 27, 2026 21:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant