Retrieval-augmented generation (RAG)
Retrieval-augmented generation (RAG) is a pattern where relevant documents are fetched from an external store at query time and inserted into the prompt, so the model answers from that fresh, specific material rather than from memory alone.
A model’s built-in knowledge is fixed at training time and cannot cover private or up-to-date data. RAG closes that gap: the user’s query is used to search a knowledge base (often via vector similarity), the top matches are pulled into the context window, and the model is asked to answer grounded in them. This makes answers current and specific, lets them cite sources, and reduces hallucination — provided the retrieval actually surfaces the right material.
Key points
- Retrieve relevant chunks at query time, put them in the prompt, then generate the answer grounded in them.
- Keeps answers current and domain-specific without retraining the model.
- Enables citations and reduces hallucination — but only as far as retrieval quality allows.
- Retrieval quality (chunking, embeddings, ranking) usually matters more than the model choice.
Learn it by building
Way of the Agent teaches AI agent engineering level by level — concepts like this one, then hands-on practice, XP, and the AI Chronicles.
Start learning free →