Document chunking strategies for RAG

Chunking determines what the retriever sees, so it determines what's retrievable. Common choices: (1) fixed-size (e.g., 512 tokens): simple but breaks mid-sentence; (2) recursive character splitting (LangChain-style): respects structure; (3) semantic chunking: split on embedding-distance breakpoints; (4) document-aware splitting (markdown headers, code function defs); (5) parent-doc retriever (small chunks for retrieval, large chunks for context); (6) hierarchical / multi-granular (summaries + chunks). The tradeoff: small chunks give precise retrieval but lose context; large chunks give context but bury the answer in noise. Best practice 2025-2026: start with markdown/structural chunking at ~256-512 tokens with 10-20% overlap, then tune with retrieval metrics (Recall@k) on a labeled eval set. Senior nuance: chunking is downstream of ingestion quality; garbage docs + a perfect chunker still gives a bad RAG.