What is the complete architecture of a production RAG system?
A production RAG system consists of five stages: ingestion, indexing, retrieval, augmentation, and generation. Ingestion: document loaders extract text from various formats (PDF, DOCX, HTML, Markdown), followed by cleaning (remove headers/footers, fix encoding, normalize whitespace) and metadata extraction (title, author, date, source URL). Indexing: text chunking splits documents into retrievable units (recursive character splitting at 512-1024 tokens with 50-100 token overlap), embedding generation converts chunks to vectors (using models like text-embedding-3-small or BGE-large-en), and vector storage indexes embeddings for similarity search (pgvector, Pinecone, Weaviate, ChromaDB). Retrieval: query embedding converts the user question to a vector, similarity search retrieves top-k candidates (typically k=5-20), optional reranking reorders candidates by relevance (using cross-encoder models like ms-marco-MiniLM), and filtering narrows results by metadata (date range, source, category). Augmentation: retrieved chunks are formatted into a prompt with the user question, system instructions define answer behavior (cite sources, acknowledge uncertainty), and context window management ensures prompt fits the model's limit. Generation: LLM produces an answer grounded in retrieved context, with post-processing for citation extraction, hallucination detection, and answer formatting. Production additions: query understanding (intent classification, query expansion), caching (store frequent query results), feedback loop (user ratings improve retrieval quality), and monitoring (track retrieval precision, answer quality, latency).