How do you implement offline vector search for local RAG systems?

Offline vector search provides embedding-based retrieval without external dependencies. Vector databases for offline use: ChromaDB (Python-native, embedded mode, supports persistent storage), LanceDB (embedded, columnar storage optimized for vector search), SQLite with sqlite-vss extension (lightweight, single-file database), and Qdrant (can run embedded or as a local server). Embedding generation: run embedding models locally using sentence-transformers library (all-MiniLM-L6-v2 for speed, BGE-large-en for quality), ONNX-optimized models for faster inference, or Ollama's embedding endpoint. Implementation: chunk documents using recursive text splitting (512-1024 tokens with 50-100 token overlap), generate embeddings for each chunk (batch processing for efficiency), store in vector database with metadata (source file, page number, section heading), and query using cosine similarity or inner product. Index types: HNSW (Hierarchical Navigable Small World) provides excellent recall with sub-millisecond search on millions of vectors (used by ChromaDB, Qdrant), IVF (Inverted File Index) trades some recall for memory efficiency, and flat index provides exact search for smaller collections. Optimization: quantize vectors from float32 to int8 for 4x memory reduction with minimal recall loss, implement metadata filtering before vector search to reduce candidate set, use hybrid search combining BM25 keyword matching with vector similarity, and cache frequent query embeddings. Scale considerations: ChromaDB handles millions of vectors on a single machine, implement collection partitioning for very large datasets, and monitor query latency (target under 50ms for interactive applications).