Fine-tuning is a great tool, but it's not the default. If your problem is mostly "use my documents," retrieval is usually a better first step.
When Fine-Tuning Makes Sense
- Consistent style/tone requirements across many outputs
- Domain-specific patterns that retrieval can't reliably provide
- Structured outputs where examples teach strong formatting behavior
LoRA vs Full Fine-Tuning
- LoRA: lower cost, faster iteration, often sufficient
- Full: higher cost, higher risk, use when LoRA can't reach target quality
Data and Evaluation
- Start with a small, high-quality dataset
- Include hard negatives and edge cases
- Hold out an evaluation set and track regressions
- Measure task success—not just generic metrics
Rule of thumb: Treat fine-tuning like adding a new dependency. It pays off when you can evaluate it continuously.