instruction tuning
Fine-tuning a pretrained model on instruction-following examples so it responds helpfully to user prompts, a key step in making GPT-3 into ChatGPT.
As defined in Nat Friedman (Github CEO) — Reading ancient scrolls, open source, & AI
Other definitions
A post-training method that trains a language model to follow natural-language instructions, which Bret cites as a major capability jump alongside ChatGPT.
Bret Taylor - The Agent Era - [Invest Like the Best, EP.386]
Where it comes up · 3
- Nat Friedman (Github CEO) — Reading ancient scrolls, open source, & AI1 claim
- Bret Taylor - The Agent Era - [Invest Like the Best, EP.386]1 claim
- 363 | Chandra Sripada on How LLMs and Humans are Cognitive Cousins1 claim
Most important claims · 3 of 3Search all claims for “instruction tuning” →
- AI progress will be fairly iterative, a gradient rather than discrete steps, with occasional large jumps such as instruction tuning in ChatGPT and GPT-4, and between jumps many incremental model improvements will occur.Bret Taylor - The Agent Era - [Invest Like the Best, EP.386]
- Next-word prediction on internet-scale data builds a rich latent representational repertoire, including latent subtask structures for complex goals, but this repertoire must be shaped by instruction tuning and reinforcement learning into useful goal-directed control.363 | Chandra Sripada on How LLMs and Humans are Cognitive Cousins
- Pretrained models can be understood as unrefined crude oil; their real capabilities only become discoverable once they are instruction-tuned and RLHF'd and deployed in usable formats—as shown by GPT-3, whose quality wasn't appreciated until instruction tuning produced ChatGPT.Nat Friedman (Github CEO) — Reading ancient scrolls, open source, & AI