What is the architecture for production-grade RAG systems at scale?

Production RAG architecture consists of five layers: ingestion, indexing, retrieval, generation, and observability. Ingestion: document processors (PDF, HTML, DOCX parsers), content extractors, and metadata enrichment, feeding into a message queue for async processing. Indexing: chunking service, embedding service (with batching and rate limiting), and vector database (Qdrant, Pinecone, Weaviate, or pgvector) with replication. Retrieval: query preprocessor, multi-strategy retriever (dense + sparse + metadata filters), reranker, and result cache (Redis). Generation: prompt assembly (system prompt + retrieved context + conversation history + user query), LLM gateway with failover between providers, streaming response handler, and output validator. Observability: trace every request through the full pipeline (use LangSmith, Arize Phoenix, or custom OpenTelemetry instrumentation), log retrieval metrics (latency, relevance scores, chunk IDs), monitor LLM costs and token usage, and track user feedback. Infrastructure: containerize each service, deploy with Kubernetes, implement horizontal scaling for retrieval and embedding services, and use CDN for static document assets. Target SLAs: P95 latency under 3 seconds, 99.9% availability.