What are common failure modes in RAG systems and how do you diagnose them?

Common RAG failure modes include retrieval failures where relevant documents exist but are not retrieved due to embedding mismatch, poor chunking, or missing metadata filters. Context failures where retrieved documents are relevant but the LLM misinterprets or ignores them, often due to poor prompt design or the lost-in-the-middle effect. Generation failures where the LLM hallucinates despite having correct context, or produces unhelpful responses. Coverage failures where the knowledge base simply does not contain the answer. Diagnosis involves logging the full pipeline at each stage: the original query, any query transformations, the retrieved documents with their scores, the complete prompt sent to the LLM, and the generated response. Compare retrieval results against expected documents for known test queries. Check embedding similarity scores to identify retrieval threshold issues. Analyze failed queries for patterns like specific topics, question types, or document sources that consistently underperform.