RAG vs fine-tuning
Retrieval changes what the model knows. Tuning changes how it behaves.
Choose RAG, retrieval-augmented generation, when the model must answer from your documents, cite them, and stay current as they change. Choose fine-tuning when you need a consistent format, tone, or narrow skill that prompting cannot reliably produce. Most business AI needs RAG first. We run RAG in NoteTube and SigmaCPM; we have not fine-tuned a language model in production.
Updated · Acasa Labs
At a glance
RAG and Fine-tuning, side by side.
| Dimension | RAG | Fine-tuning |
|---|---|---|
| What it does | RAGFinds relevant passages and gives them to the model with the question | Fine-tuningTrains the model's weights on your examples |
| Best for | RAGAnswers from documents, policies, records, and manuals | Fine-tuningOutput format, tone, classification, and narrow repeated tasks |
| Fresh knowledge | RAGUpdate the index and answers change the same day | Fine-tuningRetrain to change what it learned |
| Citations | RAGNatural: every answer can point to its source passage | Fine-tuningNot available: the knowledge is in the weights |
| Access control | RAGFilter what is retrieved per user or team | Fine-tuningEverything trained in is available to every user |
| What you need | RAGYour documents, a parsing and chunking pipeline, and a vector store | Fine-tuningHundreds to thousands of good input and output examples |
| Model choice | RAGSwap models freely; the index stays | Fine-tuningTied to the base model you tuned |
| Main failure mode | RAGRetrieving the wrong passages | Fine-tuningConfident answers the training data never supported |
| Upkeep | RAGIngestion jobs and retrieval evaluation | Fine-tuningDataset curation and retraining when the base model changes |
01
They solve different problems
RAG and fine-tuning are often presented as two ways to teach a model about your business. They are not really alternatives. Retrieval gives the model the right information at the moment it answers. Fine-tuning changes how the model responds, by training it on examples.
If the problem is "the model does not know our policies, products, or contracts", that is a knowledge problem, and retrieval is the tool. If the problem is "the model knows enough but will not answer in our format, tone, or categories", that is a behaviour problem, and better prompts or fine-tuning are the tools.
Mixing them up is expensive. A model fine-tuned on your documents still hallucinates about them, cannot tell you which page an answer came from, and is out of date the day a policy changes.
02
When RAG is the better choice
Pick RAG when answers must come from your own material: manuals, contracts, tenders, lecture notes, support history, or product data. The model reads the passages retrieved for each question, so it can quote them and you can check them.
Pick it when the material changes. Add a document to the index and it can be used in the next answer. There is no training run, and removing a document removes it from answers too.
And pick it when different users should see different things. Retrieval can be filtered by user, team, or customer before the model sees anything, which is very hard to guarantee with knowledge trained into weights.
03
When fine-tuning is the better choice
Pick fine-tuning when you need a model to produce a very consistent structure or style at volume, such as classifying tickets into your categories, extracting fields into a fixed schema, or writing in a house voice, and careful prompting with examples is still not reliable enough.
Pick it when cost or speed at high volume matters. A smaller model tuned for one narrow task can sometimes replace a larger general model, at lower cost per call and lower latency.
And pick it with good data. Fine-tuning needs a set of correct examples, typically hundreds or more, and a way to measure whether the tuned model is actually better. Without those, it is guesswork with a training bill.
04
Try these first, in this order
Start with prompting: clear instructions, a few worked examples, and structured output. Modern models follow these well, and it costs nothing to try. Then add retrieval if the model needs knowledge it does not have. Only then consider fine-tuning, for the behaviour that prompting and retrieval still cannot fix.
The two can also work together. A tuned model can be better at using retrieved passages in a particular format, while retrieval still supplies the facts. That combination makes sense for large, stable workloads, rarely for a first version.
05
What we have and have not built
Our AI work on the work page is retrieval and careful use of hosted models: grounded chat and study tools in NoteTube, cited extraction from tenders in SigmaCPM, and generation pipelines in Kraftr. We have not fine-tuned a language model for a production system. If your project needs fine-tuning, we will say so, design the evaluation, and be clear that it is not yet in our shipped work. Our founder, Pritesh Kanani, has fine-tuned 7 to 13 billion parameter models, including Dolly 2.0, LLaMA 2, and Falcon, on domain data in an earlier machine learning role, so the training and the evaluation are familiar ground.
From production
Retrieval doing the work, in production.
Answers grounded in a student's own sources
NoteTubeNoteTube parses videos, PDFs, DOCX, images, articles, and audio, chunks and embeds them into Qdrant with per-user, per-session scoping, and streams answers grounded in that material rather than the open web.
Models swapped without re-indexing
NoteTubeNoteTube lets each conversation choose a model across OpenAI, Gemini, Claude, Llama, and Mistral. The index stays the same, which a model tuned on the material could not offer.
Citations that are checked, not trusted
SigmaCPMSigmaCPM retrieves candidate pages from a tender before extraction. Every value the model returns must come with a quote found on the cited page, or it is dropped, and bond terms get a second, adversarial pass.
How we choose
The five questions we ask before choosing
Missing knowledge, citations, changing material, or per-user access point to RAG. A fixed behaviour at high volume, with good examples, points to fine-tuning, usually after prompting and retrieval.
- 01Is the problem missing knowledge, or the wrong behaviour?
- 02Must answers cite a source a person can check?
- 03How often does the underlying material change?
- 04Should different users see different documents?
- 05Do you have hundreds of correct examples to train on?
Questions
RAG vs fine-tuning, asked often.
Is RAG cheaper than fine-tuning?
Usually, to start. RAG needs an ingestion pipeline and a vector store but no training runs. Fine-tuning adds dataset work and retraining, and only pays off at high volume.
Does RAG stop hallucinations?
It reduces them and makes them checkable. The model can still misread a passage, which is why SigmaCPM verifies every quoted value against the cited page.
Can we fine-tune a model on our documents instead?
You can, but it is a poor way to add knowledge. The model will not cite sources, will mix up details, and will be out of date when documents change.
Have you fine-tuned models for clients?
Not for a production system at Acasa Labs yet. Our shipped AI work is retrieval, extraction, and generation on hosted models. Our founder fine-tuned 7 to 13 billion parameter models such as Dolly 2.0, LLaMA 2, and Falcon in an earlier machine learning role, and we will tell you plainly if your project needs fine-tuning.
Where do we start?
With a small set of real questions and the documents that answer them. We build retrieval over those, measure the answers, and decide from there.
RAG vs fine-tuning
Not sure what your AI needs? Bring the questions.
Tell us what people ask and where the answers live today. We will tell you whether this is retrieval, tuning, or just a better prompt.