Fine-Tuning — Unsloth, Axolotl, LLaMA-Factory, MLX-LM
LoRA, QLoRA, DoRA — when fine-tuning is worth the trouble vs. RAG vs. prompting.
Fine-tuning lets you adapt an open-weight model to your data. LoRA and its descendants (QLoRA, DoRA) make this affordable on consumer hardware. Unsloth, Axolotl, LLaMA-Factory, and MLX-LM are the dominant FOSS frameworks. The honest framing first: most use cases people reach for fine-tuning don't need it; RAG (rag-local) or stronger prompting solves them better.
For models see models overview; hardware at tiers; for cloud rental fine-tuning see cloud rentals.
When fine-tuning is the right call
- ★ Style / voice that prompting can't reliably hit. Marketing copy in a brand voice; in-character writing; specific code style.
- ★ Format adherence at scale. Outputting strict JSON / YAML / domain-specific structure with high reliability.
- ★ Domain language. Medical, legal, scientific jargon that the base model handles awkwardly.
- ★ Distillation. Teach a small model to mimic a larger one's behavior on your domain.
- Behavioral preferences. RLHF / DPO-style preference training for long conversations.
When fine-tuning is the wrong call
- ★ ★ "Add my company's documents to the model." Use RAG. Fine-tuning bakes facts into weights badly; RAG retrieves them precisely.
- "Make it answer questions about my data." RAG, every time.
- "Get better instruction-following." Modern instruct-tuned models are already strong; better prompts likely solve this.
- One-off task. Fine-tuning has nontrivial setup; for a single use, prompt instead.
The frameworks
★ ★ Unsloth
- ★ ★ Unsloth (github.com/unslothai/unsloth, Apache 2.0) — the dominant FOSS fine-tuning library in 2026.
- 2× faster, 70% less VRAM than reference HuggingFace TRL via custom CUDA kernels.
- Free tier supports single-GPU; paid Pro for multi-GPU and enterprise.
- Excellent Colab notebooks for every popular model — copy, edit, run.
- Pick this first for anything you can do on one GPU.
★ Axolotl
- ★ Axolotl (github.com/axolotl-ai-cloud/axolotl, Apache 2.0) — config-driven fine-tuning; the production / multi-GPU workhorse.
- YAML config defines model + dataset + training settings.
- Multi-GPU via Accelerate / DeepSpeed.
- Supports nearly every architecture and training method (SFT, DPO, ORPO, GRPO).
- Bigger learning curve than Unsloth; better at scale.
★ LLaMA-Factory
- ★ ★ LLaMA-Factory (github.com/hiyouga/LLaMA-Factory, Apache 2.0) — webui-driven fine-tuning; broad model support.
- Web UI for non-CLI users; click through the config.
- Many training methods supported.
- Popular in the Chinese-speaking AI community; English docs improving.
★ MLX-LM (Apple)
- ★ MLX-LM (github.com/ml-explore/mlx-lm, MIT) — Apple Silicon native fine-tuning via the MLX framework.
- LoRA on a 32GB+ Mac is genuinely usable.
- See Mac / Apple Silicon.
Hugging Face TRL
- The reference framework; lower-level than Unsloth / Axolotl. Use directly when you need fine control.
Other notable
- PEFT (HuggingFace) — the parameter-efficient fine-tuning library every framework builds on.
- TorchTune (PyTorch official) — newer; promising; not yet dominant.
- DeepSpeed / FSDP — distributed training engines; under most multi-GPU framework wrappers.
LoRA / QLoRA / DoRA
- LoRA (Low-Rank Adaptation) — train small adapter matrices; freeze the base model. ~1% the parameters; ~80–95% of full fine-tuning quality.
- QLoRA — LoRA on a 4-bit-quantized base model; lets a 70B fine-tune fit on a 24GB GPU.
- DoRA (Decomposed LoRA) — improvement over LoRA; small quality lift.
- Adapter merging — fold the LoRA back into base weights for deployment, or serve LoRAs separately via vLLM's multi-LoRA serving.
DPO / ORPO / GRPO — preference / reasoning training
- DPO (Direct Preference Optimization) — train on (chosen, rejected) pairs; replaces complex RLHF.
- ORPO — DPO without a separate reference model; simpler.
- GRPO (DeepSeek, late 2024) — group relative policy optimisation; the technique behind R1's reasoning training.
- All supported in Axolotl, Unsloth, LLaMA-Factory.
Hardware requirements
| Model size | LoRA | QLoRA |
|---|---|---|
| 7B | 16GB VRAM | 6GB VRAM |
| 13B | 24GB VRAM | 10GB VRAM |
| 32B | 48GB+ VRAM | 20GB VRAM |
| 70B | 80GB+ VRAM | 48GB VRAM |
| 70B with Unsloth tricks | — | 32GB VRAM (tight) |
For Tier 2 (24GB) hardware, QLoRA on 7B–13B is comfortable, 32B is borderline. Tier 3 (32GB+) opens 70B QLoRA. Tier 4 (48GB+) opens 70B LoRA at fuller precision.
For most home users: rent a H100 on vast.ai or RunPod for the duration of a fine-tune ($1–3/hr × 4–12 hours = $5–40 per fine-tune). Better economics than buying.
Dataset preparation — the unglamorous most-important step
- ★ Quality over quantity. 500 high-quality examples beats 50,000 mediocre ones.
- Format consistency. Match the chat template (Alpaca / ShareGPT / ChatML) the base model expects.
- Diversity. Avoid mode collapse — vary inputs, tasks, structures.
- Eval set. Hold out 5–10% to measure progress; iterate.
- Tools: Argilla, Lilac, Promptfoo for dataset curation and eval.
Common pitfalls
- Overfitting. Fine-tuning on a tiny dataset memorises it; you lose general capability. Mix in some general-purpose data, or use a smaller learning rate / fewer epochs.
- Catastrophic forgetting. Strong fine-tuning erases base capabilities. LoRA mitigates this; training too long doesn't.
- Wrong base model. Fine-tune the instruct version unless you specifically need the base.
- Wrong chat template. Each model family has a different template; mismatched template = silent quality disaster.
- Skipping eval. Without held-out eval you have no idea if the model improved.
Pick this if…
- You're starting and want the easy path: ★ ★ Unsloth + their Colab notebooks.
- Multi-GPU / production training: ★ Axolotl.
- You want a Web UI: LLaMA-Factory.
- Apple Silicon: MLX-LM.
- You're not sure fine-tuning is the right answer: start with RAG and stronger prompting first; only fine-tune if those genuinely don't work.
- Cost-effective fine-tune: rent H100/H200 on vast.ai for the job.