Fine-tuning takes a trained model and gives it a short second education on your own examples, your tone, your formats, your field, so the behaviour sticks without teaching anything from scratch. Think of hiring a brilliant generalist, then spending a week training them on your house style.
Where it breaks
It teaches behaviour, not facts, and people reach for it to fix the wrong problem. A model fine-tuned on your documents will start sounding like them without reliably knowing them, so answers get more confident and no more correct. And the moment your examples change or a better base model ships, you own a training pipeline you have to run again.
The number
10,000× fewer trainable parameters: reduction LoRA reports when adapting GPT-3 175B instead of retraining every weight.
Hu et al., "LoRA: Low-Rank Adaptation of Large Language Models" · 17 Jun 2021
