How do you implement fine-tuning for RAG embedding models on domain-specific data?

Fine-tuning embedding models for domain RAG: prepare training data as (query, positivepassage, negativepassage) triples. Mine negatives using BM25 (hard negatives that are keyword-similar but semantically different). Use contrastive learning objectives: MultipleNegativesRankingLoss (MNRL) for bi-encoder training with in-batch negatives, or InfoNCE loss. Training pipeline: start from a strong base model (BGE-base-en-v1.5 or E5-base-v2), prepare 5,000-50,000 training triples from your domain, train for 3-10 epochs with learning rate 2e-5, batch size 32-128, and evaluate on a held-out set using NDCG@10. Data generation strategies: use an LLM to generate questions from your document chunks (synthetic query generation), leverage user query logs paired with clicked/viewed documents, and manually annotate 200+ query-document relevance pairs for evaluation. Tools: sentence-transformers library, or use Matryoshka representation learning for flexible dimension embeddings. Expected improvement: 15-30% retrieval accuracy boost on domain-specific queries. Fine-tuning typically takes 1-4 hours on a single GPU and the investment pays for itself through dramatically better retrieval quality.