pagedattention

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

An Efficient and Versatile Inference Engine for Distributed LLM Serving

面向开发者与初学者的 nano-vLLM 交互式源码教程 - 通过 13 个 HTML 互动实验 + 13 章中文教程理解大模型推理引擎

Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models.

LLM inference kernels from scratch in Triton: KV cache, FlashAttention, PagedAttention, RMSNorm, RoPE, SwiGLU, and benchmarks.

A deterministic PyTorch autograd verification trap for catching silent KV-cache routing and block-alignment failures in vLLM and SGLang serving infrastructure.

Logical KV cache memory profiler, zombie leak detector, and fragmentation inspector for PagedAttention (vLLM, SGLang)

Speed-Optimized LLM Inference Server (L1-L7 full stack: FlashAttention, PagedAttention, AWQ INT4, Speculative Decoding, Continuous Batching, and FastAPI serving)

Deadline-aware KV-cache scheduling for protecting decode-critical request-state under long-context LLM inference pressure.

Disaggregated Prefill-Decode serving engine connecting NVIDIA Blackwell (SM120) and Huawei Ascend 910B2 (CANN) across physical nodes.

Empirical benchmarking harness mapping the boundary between compute saturation and PagedAttention KV-cache preemption cascades under streamed test-time compute scaling.

From-scratch model of an LLM serving engine's systems core: paged KV-cache, continuous batching, preemption, and prefix caching — GPU-free, with reproducible benchmarks.

French course summary of “Fast & Efficient LLM Inference with vLLM,” covering inference, quantization, LLM Compressor, PagedAttention, Continuous Batching, benchmarking, and evaluation.

LLM inference engine internals: PagedAttention, RadixAttention, continuous batching, speculative decoding - kernels to distributed systems

⚡ Production vLLM LLM inference serving architecture featuring PagedAttention, continuous batching, and Docker deployment

Paged KV cache inference server. Go control plane, Python worker. 97% pool utilization vs 22% baseline, 4.4x concurrency, output bit-identical to HuggingFace.

LLM inference serving prototype with continuous batching, paged KV-cache management, benchmark telemetry, and C++/ONNX Runtime gRPC backend scaffold.

Considerations about running AI Inference workloads on Kubernetes

A minimal LLM inference engine implementing PagedAttention-style KV cache management on NanoGPT. Based on the "Efficient Memory Management for Large Language Model Serving with PagedAttention" paper.

Hand-written Triton PagedAttention decode kernel with reproducible correctness & performance benchmarks against PyTorch baselines