Curiosity · Concept

Instruction Tuning (SFT)

Instruction tuning — also called supervised fine-tuning or SFT — teaches a base LLM to behave like an assistant. You gather or generate high-quality (prompt, completion) pairs covering diverse tasks, then fine-tune the base model on them with standard next-token cross-entropy. The result is a model that follows instructions instead of just continuing prose.

Quick reference

Proficiency
Intermediate
Also known as
SFT, supervised fine-tuning, instruction fine-tuning
Prerequisites
Fine-tuning, Tokenization

Frequently asked questions

What is instruction tuning?

It is the supervised fine-tuning stage that teaches a pretrained model to follow instructions. You train on thousands of (instruction, ideal response) examples so the model learns that 'Summarize this article' should produce a summary, not more article text.

How is instruction tuning different from RLHF?

Instruction tuning (SFT) uses standard supervised learning on demonstrations — you show the model what a good answer looks like. RLHF uses preference data (A is better than B) to refine the SFT model further. Modern pipelines do SFT first, then preference alignment via DPO or RLHF.

What are the most common instruction-tuning datasets?

FLAN collection (Google), Alpaca and Vicuna (Stanford/LMSYS — distilled from GPT-3.5/4), OpenOrca, Dolly (Databricks), UltraChat (Tsinghua), Tulu (AI2), and enterprise-specific mixtures. Many modern recipes use carefully filtered synthetic data from stronger models.

How much data does instruction tuning need?

Less than you'd think. LIMA (Meta, 2023) showed that 1,000 carefully curated examples can produce a strong assistant. Quantity helps, but quality and diversity matter more — noisy examples can actively hurt.

Sources

  1. Wei et al. — Finetuned Language Models Are Zero-Shot Learners (FLAN) — accessed 2026-04-20
  2. Zhou et al. — LIMA: Less Is More for Alignment — accessed 2026-04-20
  3. Stanford Alpaca — accessed 2026-04-20

Related

  • Direct Preference Optimization (DPO) — Direct Preference Optimization (DPO) is an alignment technique that fine-tunes a language model directly on pairs of preferred vs dispreferred responses, skipping the reward model and RL loop used in RLHF.
  • Fine-tuning — Fine-tuning adapts a base LLM's weights to new task formats, style, or tone using labeled examples. Prefer RAG for new facts; fine-tune for new behavior.
  • Reinforcement Learning from Human Feedback (RLHF) — RLHF is the training technique that aligns a language model's behavior with human preferences by using human-ranked outputs to train a reward model, then fine-tuning the LLM against that reward with reinforcement learning.
  • All Concepts — Browse the complete concepts library
  • VSET — Programmes offered — B.Tech programs (CSE, AI&ML, AI&DS, Electronics, IIoT)