Fine-tuning vs LoRA vs RAG
Three ways to specialize an LLM — and when each one is the right call.
Full fine-tuning retrains all the weights on new data — powerful but expensive and static. LoRA freezes the base and trains small low-rank adapters — cheap, fast and swappable. RAG changes no weights at all, injecting fresh knowledge at inference via retrieval. Rule of thumb: RAG for knowledge and freshness, fine-tuning/LoRA for behavior, format and skill.
Full fine-tuning
Continue training the base model on your own data, updating every one of its billions of weights. This reshapes behavior, tone and skill most deeply, effectively teaching the model new habits. But it's the most expensive option in compute and expertise, produces a full-size copy per use case, and freezes whatever knowledge it absorbs the moment training ends.
LoRA
Low-Rank Adaptation freezes the entire base model and trains only small pairs of low-rank adapter matrices injected alongside it. You update a tiny fraction of the parameters, so it's dramatically cheaper and faster than a full fine-tune while capturing most of the benefit. The adapters are small files you can store, share, and hot-swap on top of one shared base.
RAG
Leave the model's weights completely untouched and instead supply knowledge at inference time by retrieving relevant documents and inserting them into the prompt. Updating what the model knows is as simple as changing the underlying documents — no training run at all. It's the cheapest and fastest way to keep facts current and to ground answers in citable sources.
Choosing
The deciding question is whether you're changing what the model knows or how it behaves. For fresh, factual, or proprietary knowledge, reach for RAG; for tone, format, or a specialized skill, reach for fine-tuning or LoRA. They aren't mutually exclusive — a common production pattern is a LoRA-tuned model for voice paired with RAG for current facts.
The real question: knowledge or behavior?
Once you have a capable base LLM, you almost always need to specialize it for your use case. A generic model doesn't know your company's private documents, doesn't reliably answer in your required format, and doesn't speak in your brand's voice. There are three main tools for closing that gap — full fine-tuning, LoRA, and RAG — and the single biggest source of confusion in applied AI is treating them as interchangeable. They are not. They change fundamentally different things.
The clarifying question to ask before anything else is: am I trying to change what the model knows, or how the model behaves? This one distinction cuts through most of the decision.
- Knowledge is facts and information — the content of an answer. "What is our refund policy?" "What did the Q3 report say?" "Which API endpoint does this?" Knowledge changes constantly and needs to be accurate and current.
- Behavior is the model's manner and skill — the *shape* of an answer regardless of content. Its tone, its output format (always valid JSON, always a certain structure), its style, its ability to perform a specialized task the way an expert would.
RAG is a knowledge tool: it injects information at inference time without touching the model. Fine-tuning and LoRA are behavior tools: they adjust the model's weights so new habits and skills are baked in. Get this mapping right and the choice between approaches largely makes itself. The rest of this article explains how each works, what it costs, and how to combine them.
Full fine-tuning: rewriting the weights
Full fine-tuning takes a pre-trained base model and continues training it on your own dataset, updating *all* of its parameters — potentially billions of weights — through more gradient descent. You're not training from scratch; you're nudging an already-capable model toward your data. This is the most powerful way to change behavior: give it thousands of examples in your desired format or style and it will deeply internalize them, effectively acquiring new habits and skills rather than being told about them in a prompt.
That power comes at a steep price on several axes. Compute cost is high — updating every weight of a large model requires serious GPU resources and time, plus the memory to hold the model, its gradients, and optimizer state. Expertise is required to curate a good dataset and avoid pitfalls. Storage balloons: each fine-tuned model is a full-size copy of the base, so ten use cases means ten complete multi-gigabyte models to host and serve.
And there are two failure modes to respect. The first is catastrophic forgetting: pushing hard on narrow new data can degrade the general capabilities the base model already had, making it worse at everything outside your dataset. The second is that fine-tuning bakes in static knowledge — whatever facts appear in the training data are frozen at training time, can't be updated without retraining, and can't be cited. Fine-tuning is superb for behavior and skill; it is a poor and expensive way to inject facts. That expense is exactly what LoRA was invented to reduce.
LoRA: most of the benefit, a fraction of the cost
LoRA (Low-Rank Adaptation) is the most popular form of parameter-efficient fine-tuning, and its core idea is clever. Instead of updating the base model's weights, you freeze them entirely and insert small, trainable low-rank adapter matrices alongside the layers you want to influence. During training, only these tiny adapters learn; the enormous base is left untouched.
Why "low-rank"? The insight is that the *change* a fine-tune needs to make to a big weight matrix can be well approximated by the product of two much smaller matrices. Rather than learn a full update the size of the original weight, LoRA learns those two skinny factors, which together have a tiny fraction of the parameters. At inference the adapter's contribution is added to the frozen base's output. The practical consequences are large:
- Cheap and fast to train — you're optimizing a fraction of a percent of the parameters, so it needs far less GPU memory and time.
- Tiny artifacts — a trained adapter is often just a few megabytes, versus gigabytes for a full model.
- Swappable — because every adapter sits on top of the *same* frozen base, you can host one base model and hot-swap adapters for different tasks or customers, or even compose them. Serving a hundred specialized behaviors no longer means a hundred full models.
The trade-off is that LoRA is slightly less expressive than a full fine-tune — for the most demanding behavioral changes, updating every weight can still edge it out. But for the vast majority of real use cases, LoRA captures most of the benefit at a small fraction of the cost, which is why it has become the default way teams fine-tune. Note that LoRA is still a fine-tuning technique: like full fine-tuning, it changes *behavior*, and its knowledge is still static.
RAG as the third option, and how to choose
RAG (Retrieval-Augmented Generation) is the odd one out: it changes no weights at all. Instead, at inference time it retrieves relevant documents from an external store and inserts them into the prompt, so the model answers from real, supplied text rather than its frozen memory (the companion RAG concept covers the mechanism in full). Because knowledge lives in an external index rather than the weights, updating it is trivial — edit a document, re-index, done — and answers can cite their sources. RAG is the cheapest, freshest, most auditable way to give a model knowledge it lacks.
Line the three up on the trade-offs that matter:
- What it changes: RAG changes *knowledge*; fine-tuning and LoRA change *behavior and skill*.
- Update cost/freshness: RAG updates instantly by changing documents. LoRA requires a (cheap) retrain. Full fine-tuning requires an expensive retrain — both bake knowledge in statically.
- Upfront cost: RAG is low (build an index); LoRA is moderate; full fine-tuning is high.
- Inference cost/latency: RAG *adds* latency and tokens at query time (the retrieval hop plus injected context). Fine-tuning and LoRA add nothing at inference — the behavior is already in the weights.
- Citations/auditability: RAG can cite; tuned models cannot.
So the decision rule: need current, factual, or proprietary knowledge → RAG. Need a specific tone, format, or specialized skill → LoRA (or full fine-tuning if LoRA isn't enough). And critically, these are not either/or. The most robust production systems combine them: fine-tune or LoRA-tune a model so it reliably behaves the right way — the correct voice, structure, and task competence — and layer RAG on top so it always answers with fresh, grounded, citable facts. Pick the tool by what you're actually trying to change, and reach for more than one when you need to change both.
Full fine-tuning retrains every weight on new data: it changes behavior and style but is expensive and its knowledge is frozen at training time.
LoRA is parameter-efficient fine-tuning: freeze the base weights, train small low-rank adapter matrices — cheap, fast, and swappable.
RAG changes no weights; it injects fresh external knowledge at inference through retrieval, and is the cheapest to keep up to date.
For changing facts and freshness, reach for RAG. For changing behavior, format or skill, reach for fine-tuning or LoRA.
They combine well: a LoRA-tuned model for tone paired with RAG for current facts is a common production setup.
Quick quiz
Which approach is best when your facts change frequently?
What defines LoRA compared to full fine-tuning?
What is Retrieval-Augmented Generation (RAG)
Ground an LLM in your own data so it stops making things up.
How the Transformer architecture works
The architecture behind GPT, Claude, Gemini and Llama — one block at a time.
How AI agents chain tools, memory & reasoning
An LLM in a loop that plans, acts, observes — and keeps going until it's done.