Instruction Tuning

Raw pre-trained models complete text; they do not follow instructions. Instruction tuning bridges this gap by training the model on a dataset of (instruction, desired response) pairs: "Summarize this article in 3 bullet points → [bullet points]", "Fix the bug in this Python function → [corrected code]". After instruction tuning, the model generalizes to follow novel instructions in zero-shot settings.

FLAN (Finetuned Language Net, Wei et al., 2022) demonstrated that instruction tuning at scale transfers across tasks: a model trained on thousands of diverse instruction types improves on held-out task types it has never seen. The diversity and quality of the instruction dataset matters more than its size beyond a certain threshold.

Modern instruction tuning typically uses synthetic data generated by stronger models (a technique sometimes called "self-play" or "constitutional synthetic data generation"). Anthropic, Meta, and Mistral all use synthetic instruction data alongside human-written examples to build the instruction-following capabilities of their production models.

## Related Terms

- [[glossary/pre-training|Pre-training]] — the large-scale unsupervised training run that teaches the model language and world knowledge - [[glossary/supervised-fine-tuning|Supervised Fine-Tuning]] — the weight-update process that adapts a pre-trained model to a target task using labeled data - [[glossary/fine-tuning|Fine-tuning]] — the weight-update process that adapts a pre-trained base model for downstream tasks - [[glossary/rlhf|Reinforcement Learning from Human Feedback (RLHF)]] — the training technique that uses human preference comparisons to steer model behaviour - [[courses/gemini-enterprise-agents|Course: Gemini Enterprise Agents]] — hands-on practice with the concepts covered in this entry