What is contextual retrieval and how does it improve RAG accuracy?

Contextual retrieval, introduced by Anthropic, prepends contextual information to each chunk before embedding. The technique: for each chunk, use an LLM to generate a concise context statement explaining how the chunk fits within the broader document (e.g., "This chunk is from the Q3 2024 earnings report, specifically discussing revenue from the cloud services division"). This context is prepended to the chunk text before embedding and storage. Impact: contextual retrieval reduces retrieval failures by up to 49% when combined with hybrid search. Implementation: during indexing, batch-process chunks with their surrounding document context through a fast LLM, cache the contextual prefixes, and embed the combined text. Cost: the one-time LLM cost during indexing is typically less than $1 per 1000 document pages using cached prompt prefixes. For large document collections, use prompt caching to reduce costs by 90%. Combine with BM25 hybrid search for maximum benefit. This technique is particularly effective for chunks that are ambiguous without context, such as tables, list items, or paragraphs that reference information from earlier in the document.