How do you evaluate RAG system performance?

RAG evaluation covers three dimensions: retrieval quality, generation quality, and end-to-end system performance. Retrieval metrics include recall at k measuring whether relevant documents appear in the top k results, mean reciprocal rank for ranking quality, and precision measuring the ratio of relevant to total retrieved documents. Generation metrics include faithfulness measuring whether the answer is supported by the retrieved context, relevance measuring how well the answer addresses the query, and completeness measuring whether all important aspects are covered. End-to-end metrics include answer correctness compared to ground truth, latency, and user satisfaction scores. Frameworks like RAGAS, TruLens, and DeepEval provide automated evaluation pipelines. Building a golden test set of query-answer-context triples enables reproducible benchmarking across system changes. Regular human evaluation supplements automated metrics for subjective quality dimensions.