[lzhangzz]

Summary

Adds speculative decoding to the TurboMind backend, built on a new fixed-chain speculator framework with two draft methods:

• mtp — Qwen3.5 MTP draft layers, weights read directly from the target checkpoint
• eagle3 — Qwen3 EAGLE3 with a sidecar draft model; added to bootstrap the MTP integration, not optimized for performance (linear chain verification only, no tree verification)

What's included

Draft model framework (src/turbomind/models/speculative/)

• Fixed-chain speculator base: model/policy/setup split, hidden-state tap, and a method-keyed draft weight registry
• eagle3/: EAGLE3 draft model, weight mapping, and target hidden-state projection with dedicated kernels — serves as the integration vehicle for the speculative pipeline rather than a performance-optimized path; verification is linear-chain only, no tree verification
• qwen3_5_mtp/: Qwen3.5 MTP draft model, weights, and target final-hidden extraction

Verification path

• Block speculative verification (generation/target_verification)
• Dedicated verification attention kernels (kernels/attention/verification/): SM80 path, SM90 WGMMA pipelines, paged KV, and cross-CTA reduce
• SM90 GDR verify-and-commit kernel mode with persistent pipelining
• Op kernels for draft carry, speculative sampling, and speculative sequence bookkeeping, all with python bindings

Engine and scheduler refactor

• Unified, parameterized batch-op pipeline driven engine-side
• Executor forward split into target pass and speculative round
• Model (formerly BatchComponents) owns target, vision, and speculative models
• Submitted rows carry producer-set effects instead of a kind tag
• Unified required admission with simplified rollback
• Normative contract expanded in src/turbomind/engine/README.md

Python surface (lmdeploy/)

• spec_decode.py method registry plus qwen3_5_mtp / qwen3_eagle3 model and weight builders, wired through speculative_config
• Metrics and async-engine support for speculative stats

Tests

• Op-level pytest suites comparing against pure-PyTorch references: verification attention, draft carry, speculative sampling, speculative sequence, target hidden projection (232 collected cases), plus a copy-op suite
• scripts/test_turbomind_model.py extended for speculative smoke testing

Usage

from lmdeploy import TurbomindEngineConfig
from lmdeploy.messages import SpeculativeConfig

# Qwen3.5 MTP (draft weights come from the target checkpoint)
spec = SpeculativeConfig(method='mtp', num_speculative_tokens=2)

# Qwen3 EAGLE3 (sidecar draft model)
spec = SpeculativeConfig(method='eagle3',
                         model='<path/to/eagle3-draft>',
                         num_speculative_tokens=3)

engine = TurbomindEngineConfig(speculative_config=spec)