Project 2: Fine-tuning & LoRA
You almost never train a model from scratch (last chapter showed why: it takes millions of dollars). Instead you take a pretrained model and fine-tune it: a little extra training to adapt its broad knowledge to your task. And the modern, efficient way to do that (the technique behind nearly every custom LLM today) is LoRA. This project builds LoRA from scratch and measures why it's a big deal.
Full code: code/projects/finetune_lora.py, listed in full at the end of this chapter.
What fine-tuning is
A pretrained model already learned general features (language, edges, concepts) from a huge corpus. Fine-tuning continues training it on a smaller, task-specific dataset so it specializes, sentiment on your reviews, your company's writing style, a medical domain. It's transfer learning: reuse the expensive pretraining, pay only for the cheap adaptation.
The problem: full fine-tuning updates all the weights. For a 70-billion-parameter model that means storing and training 70 billion numbers, enormous memory and a full copy of the model per task. LoRA fixes exactly this.
The LoRA idea: freeze the giant, train a tiny add-on
LoRA (Low-Rank Adaptation) freezes the pretrained weights $W_0$ and adds a small trainable detour: a low-rank (Chapter 23) pair of matrices $A$ and $B$. The adapted layer is:
$$ y = x W_0 + \frac{\alpha}{r}, x A B $$
where $A$ is $(d \times r)$ and $B$ is $(r \times d)$ with the rank $r$ tiny (4, 8, 16). You train only $A$ and $B$ (a handful of parameters), while the huge $W_0$ never moves. From the project:
class LoRALinear(nn.Module):
def __init__(self, base, r=4, alpha=8):
self.base = base
for p in self.base.parameters():
p.requires_grad = False # FREEZE the pretrained weights
self.A = nn.Parameter(torch.randn(base.in_features, r) * 0.01)
self.B = nn.Parameter(torch.zeros(r, base.out_features)) # starts as a no-op
self.scale = alpha / r
def forward(self, x):
return self.base(x) + self.scale * (x @ self.A @ self.B) # frozen + low-rank
Why it works: the change needed to adapt a model is usually low-rank: it lives in a small subspace.
Bstarts at zero so training begins exactly at the pretrained model and only departs as needed. This is the same low-rank, "best approximation with few numbers" idea as SVD (Chapter 23).
Measuring the win
The project adapts a "pretrained" layer to a new classification task two ways, full fine-tuning vs. LoRA, and counts trainable parameters:
$ python3 finetune_lora.py
Output:
full fine-tune : trainable params = 1188 acc = 1.000
LoRA (r=4) : trainable params = 388 acc = 1.000
LoRA trains 3.1x fewer parameters for comparable accuracy.
Same 100% accuracy, a third of the trainable parameters. On our toy layer the ratio is 3×; on a real LLM, where the frozen base is billions of weights and the adapter is millions, the ratio is 100 to 1000×. That's the difference between needing a data center and fine-tuning on a single GPU.
Why LoRA changed the game
- Cheap: train <1% of the parameters; fits on modest hardware.
- Small artifacts: a LoRA adapter is a few megabytes, not gigabytes. Ship one base model + many tiny adapters (one per customer/task) instead of many full copies.
- No quality loss: matches full fine-tuning on most tasks.
- Composable & reversible: swap adapters in and out; the base is untouched.
QLoRA adds quantization (Chapter 27) so you can fine-tune a 70B model on a single consumer GPU. This is why a small team can now customize a giant model.
The real recipe (follow-along)
In practice you don't hand-roll LoRALinear: you use Hugging Face transformers +
peft. The whole production fine-tune is a few lines:
# pip install transformers peft datasets
from transformers import AutoModelForSequenceClassification, TrainingArguments, Trainer
from peft import LoraConfig, get_peft_model
model = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased")
lora = LoraConfig(r=8, lora_alpha=16, target_modules=["q_lin", "v_lin"])
model = get_peft_model(model, lora) # wraps the model: freezes base, adds LoRA
model.print_trainable_parameters() # e.g. "trainable: 0.6% of all params"
Trainer(model, TrainingArguments("out", num_train_epochs=3),
train_dataset=ds).train() # trains ONLY the adapter
model.save_pretrained("my-adapter") # a few MB, not GB
get_peft_model does exactly what our LoRALinear did, freeze the base, inject
low-rank adapters into the attention layers, just for a real pretrained transformer.
Don't be confused: fine-tuning vs. RAG vs. prompting. Fine-tuning changes the model's behavior/skill by training (use it for tone, format, a narrow capability). RAG (Chapter 14) adds knowledge at query time without training. Prompting changes nothing. To make a model act a certain way → fine-tune; to make it know your latest docs → RAG. Reaching for fine-tuning when you needed RAG (or vice versa) is the most common modern-AI design mistake.
Make it production
- Track every fine-tune with MLflow; version the adapter in a registry (tools book).
- Evaluate before shipping: a held-out set and an eval gate (Chapter 10; the tools book's testing/eval chapters).
- Serve the base model once and hot-swap adapters per task.
Full source: code/projects/finetune_lora.py
The complete, runnable program, exactly as verified for this chapter:
"""
finetune_lora.py — fine-tuning, and LoRA, from scratch.
Fine-tuning = take a pretrained model and adapt it to a new task. The modern,
efficient way is LoRA (Low-Rank Adaptation): FREEZE the big pretrained weights and
train only a tiny pair of low-rank matrices added alongside them. You get ~full
fine-tuning quality while training a fraction of the parameters.
This demo makes the idea concrete and measurable: we "pretrain" a feature layer,
freeze it, then adapt it to a new task two ways — full fine-tuning vs. a LoRA
adapter — and compare trainable parameter counts and final accuracy.
Run (needs torch; CPU is fine):
python finetune_lora.py
"""
from __future__ import annotations
import torch
import torch.nn as nn
import torch.nn.functional as F
torch.manual_seed(0)
DIM, N_CLASSES = 32, 4
def make_task(n=600):
"""A synthetic classification task: 4 classes from 4 Gaussian clusters."""
centers = torch.randn(N_CLASSES, DIM)
y = torch.randint(0, N_CLASSES, (n,))
X = centers[y] + 0.6 * torch.randn(n, DIM)
return X, y
class LoRALinear(nn.Module):
"""A frozen base Linear with a trainable low-rank update: y = xW0 + (alpha/r)·xA B."""
def __init__(self, base: nn.Linear, r: int = 4, alpha: int = 8):
super().__init__()
self.base = base
for p in self.base.parameters(): # FREEZE the pretrained weights
p.requires_grad = False
self.A = nn.Parameter(torch.randn(base.in_features, r) * 0.01)
self.B = nn.Parameter(torch.zeros(r, base.out_features)) # start as a no-op
self.scale = alpha / r
def forward(self, x):
return self.base(x) + self.scale * (x @ self.A @ self.B)
def trainable(m):
return sum(p.numel() for p in m.parameters() if p.requires_grad)
def train(model, X, y, steps=300, lr=0.05):
opt = torch.optim.Adam((p for p in model.parameters() if p.requires_grad), lr=lr)
for _ in range(steps):
loss = F.cross_entropy(model(X), y)
opt.zero_grad(); loss.backward(); opt.step()
acc = (model(X).argmax(1) == y).float().mean().item()
return acc
def main():
X, y = make_task()
# A "pretrained" feature layer (imagine it was trained on a big corpus).
pretrained = nn.Linear(DIM, DIM)
# --- Option A: full fine-tuning (train a head + the whole base copy) ---
full = nn.Sequential(nn.Linear(DIM, DIM), nn.ReLU(), nn.Linear(DIM, N_CLASSES))
full[0].load_state_dict(pretrained.state_dict()) # start from pretrained
acc_full = train(full, X, y)
print(f"full fine-tune : trainable params = {trainable(full):4d} acc = {acc_full:.3f}")
# --- Option B: LoRA (freeze the base, train only the low-rank adapter + head) ---
lora_base = LoRALinear(nn.Linear(DIM, DIM), r=4, alpha=8)
lora_base.base.load_state_dict(pretrained.state_dict())
lora = nn.Sequential(lora_base, nn.ReLU(), nn.Linear(DIM, N_CLASSES))
acc_lora = train(lora, X, y)
print(f"LoRA (r=4) : trainable params = {trainable(lora):4d} acc = {acc_lora:.3f}")
ratio = trainable(full) / trainable(lora)
print(f"\nLoRA trains {ratio:.1f}x fewer parameters for comparable accuracy.")
if __name__ == "__main__":
main()
The takeaway
Fine-tuning adapts a pretrained model to your task; LoRA does it efficiently by freezing the base and training a tiny low-rank adapter: you measured 3× fewer parameters here, 100 to 1000× on real LLMs, at no accuracy cost. It's the same low-rank idea as SVD, and the reason custom LLMs are now within reach of small teams. Next, we give a model the ability to act: an agent with tools. 👉