factual
30 examples fixed model self-knowledge broadly
Early ChatGPT models wrongly claimed capabilities they lacked (e.g. sending emails, calling an Uber); a tiny dataset of around 30 examples showing the model doesn't have those capabilities, mixed into the training data, generalized well to fix this across all sorts of capabilities they didn't explicitly train for.
factualpending
Evidence Quote
“something like 30 examples. We had a pretty small number of examples showing this general behavior... That generalized pretty well to all sorts of capabilities we didn't train for.”
Source
John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:34:30 AM
Your notes
Loading notes...