How do you implement multi-step or iterative retrieval in RAG?

Multi-step retrieval addresses complex queries that cannot be answered from a single retrieval pass. The process starts by decomposing the original query into sub-questions, retrieving context for each, and iteratively building a comprehensive answer. Implementation approaches include chain-of-thought retrieval where the model reasons about what additional information it needs and formulates follow-up queries, tree-of-retrieval where multiple retrieval paths are explored in parallel and the best results are combined, and IRCoT or Interleaving Retrieval with Chain of Thought which alternates between retrieval and reasoning steps. Frameworks like LlamaIndex provide built-in sub-question query engines and recursive retrieval agents. This pattern is particularly valuable for complex analytical questions, comparative queries, and multi-hop reasoning tasks where the answer depends on connecting information from multiple documents.