/ all insights
AI·Apr 14, 2026·10 min read
RAG vs fine-tuning: a decision tree you can actually use.
Six questions to ask before you spend a week on the wrong approach.
The choice between retrieval and fine-tuning should start with the failure mode. If the model lacks the right facts, retrieval is usually the first move. If it has the facts but keeps responding in the wrong format, tone, or decision pattern, fine-tuning may eventually help.
Start with the question the system gets wrong
| Failure | Likely fix | Why |
|---|---|---|
| The answer needs private or changing company data. | RAG | The model needs fresh context at request time. |
| The answer must cite policies, docs, invoices, or tickets. | RAG | Retrieval gives the system inspectable evidence. |
| The model knows the facts but ignores your output schema. | Prompting and validation first | Most format issues can be caught without training. |
| The model must imitate a consistent judgement pattern from many examples. | Fine-tuning later | Training can help once examples and eval criteria are clear. |
| The workflow needs tool calls and state changes. | Application design | The hard part is orchestration, not model memory. |
A practical decision path
- Use retrieval when the knowledge changes or belongs to the business.
- Use structured prompts and validators when the problem is output shape.
- Use evals before fine-tuning so you know what improvement means.
- Fine-tune only after you have enough examples, stable labels, and a measurable behavior gap.
- Keep the source data and evaluation set separate so the system can be audited.