Definition
Retrieval-augmented generation (RAG)Retrieval-augmented generation (RAG) is a technique where an AI model first searches a set of documents for passages that match a question, then writes its answer from those passages. Because it looks things up at answer time, the model can use up-to-date information and point to where it came from.
How RAG works
Retrieval-augmented generation runs every time someone asks a question, and it has three stages.
- Retrieve. The system searches a document collection for the passages most relevant to the question. The collection might be a company knowledge base, a pile of product manuals, or the web. Many systems match by meaning using embeddings, often alongside ordinary keyword search.
- Augment. The best passages go into the prompt next to the original question, usually with an instruction like "answer using these sources." Everything has to fit in the model's context window.
- Generate. The language model writes its answer from that longer prompt. Many systems also add citations pointing back to the passages they used.
Before any of this can happen, documents get split into smaller pieces. That step is content chunking, and it's the reason AI engines tend to quote a paragraph or a section instead of a whole page. Chunking is also how RAG handles long documents: a model can only read a limited amount of text at once (its context window), so only the most relevant chunks go into the prompt.
The name comes from a 2020 paper, "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks," by Patrick Lewis and colleagues at Facebook AI Research, presented at NeurIPS 2020. They connected a language model to a searchable index of Wikipedia and found the combined system wrote more specific and more factual answers than the model did on its own.
Why RAG exists
Language models have two limits that retrieval works around.
Their knowledge stops at a date. A model only knows what was in its training data, up to its knowledge cutoff, so anything published later has to be fetched.
They also know nothing private. A model trained on public text has never seen your pricing sheet or your support docs, and retrieval gives it access to them without retraining the model.
There's a third benefit: answers become checkable. When a model cites the passage it used, a reader can open the source and confirm the claim. That link between an answer and its source is called grounding.
RAG vs a plain LLM
| Plain LLM | RAG system | |
|---|---|---|
| Source of knowledge | Training data only | Training data plus retrieved documents |
| Freshness | Fixed at the knowledge cutoff | As current as the document collection |
| Citations | None, or unreliable | Can link to the passages it used |
| Private data | Not available | Available if it's in the collection |
The model can be identical in both columns. What changes is the material in front of it when it writes.
RAG vs fine-tuning
Both adapt a model to a job, and they change different parts of the system.
Fine-tuning retrains the model on extra examples, so the new behavior lives in the model's weights. It suits teaching a tone or a fixed output format, and updating what the model knows means training it again.
RAG leaves the model alone and supplies facts when the question arrives. It suits information that changes often or needs a source attached, and updating it means editing the documents.
Plenty of production systems use both, with a fine-tuned model handling tone and format and retrieval handling the facts.
RAG and hallucinations
Retrieval lowers the rate of AI hallucinations because the model has real text to work from, though some still get through. A model can misread a passage, merge details from two sources, or add something no source said. And if retrieval returns an outdated or inaccurate page, the answer inherits the mistake.
This explains a lot of the wrong facts brands find in AI answers. The engine often did retrieve a real page; that page was just out of date, or described the product badly.
What RAG means for brands
AI search engines work like RAG systems running at web scale. Perplexity searches the web in real time when you ask a question. ChatGPT searches when it decides current information would help, and you can also turn search on yourself. Google says AI Overviews and AI Mode may run several related searches (query fan-out) and show supporting links from the web. In every case, an answer can only cite pages the system retrieved first.
For your content, that comes down to a few practical checks:
- Can it be found? The page has to be crawlable and indexed, and it has to match the searches the engine runs, including the related queries it generates from one question.
- Does each section stand on its own? A clear heading with the key point near the top lets a single passage be lifted without the rest of the page.
- Is it specific enough to quote? A passage that answers the question directly, with sourced facts, is easier for a model to use than a vague one.
Retrieval is also why the sources an engine cites shift from week to week. Our guide to how AI answer engines source content covers that process in more detail.
The only way to know whether these engines retrieve and cite your pages is to track their answers. Searchable records which URLs AI engines such as ChatGPT and Perplexity cite for your tracked prompts, so you can see which of your pages get used and which competitor pages get picked instead. For the metric itself, see AI citations.
Frequently asked questions
RAG stands for retrieval-augmented generation. The system retrieves relevant documents, adds them to the model's prompt, and the model generates its answer from them.
See how AI engines talk about your brand
Get a free report on where ChatGPT, Perplexity, and Google AI name and cite you.