What causes hallucination in RAG systems and how do you mitigate it?

RAG hallucination occurs when the LLM generates information not supported by the retrieved context. Causes include insufficient relevant context in the retrieval results, the LLM relying on parametric knowledge instead of the provided context, poorly formatted context that the model struggles to parse, and queries that fall outside the knowledge base coverage. Mitigation strategies include improving retrieval quality through better chunking, embedding models, and reranking. Use explicit instructions in the system prompt requiring the model to only answer based on provided context and to say it does not know when context is insufficient. Implement citation requirements forcing the model to reference specific source passages. Apply answer verification by checking generated claims against retrieved documents. Use smaller focused context windows rather than dumping large amounts of marginally relevant text. Monitor and log hallucination rates for continuous improvement.