Paged attention (vLLM)

Paged attention (Kwon et al., SOSP 2023) borrows virtual-memory paging for KV cache. Traditional KV cache allocates one contiguous block per sequence: heavy fragmentation wastes memory. Paged attention stores the KV cache in fixed-size "pages" (e.g., 16 tokens per page), indexed by a per-sequence page table. Memory is allocated only as needed and pages are reused across sequences when possible. Empirically this gives 2-4x throughput over naive static allocation (HuggingFace transformers) and 14-24x over FasterTransformer in multi-tenant serving. vLLM is the open-source implementation. Senior nuance: paged attention unlocks continuous batching, prefix-sharing across requests, beam-search efficient caching, and serves as the foundation for modern LLM serving alongside SGLang, TGI, TensorRT-LLM.