causal
Helpfulness preference model generalizes to STEM without STEM data
You don't need closely matched labels or domain experts for every capability: the base model has already seen documentation, code, shell scripts and man pages, so a few-shot prompt can elicit answers like FFmpeg usage, and training a preference model purely for helpfulness generalizes to STEM and programming even without any STEM or programming examples.
causalpending
Evidence Quote
“training a preference model for helpfulness will, even if you don't train it on any STEM, somewhat generalize to STEM.”
Source
John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:34:30 AM
Your notes
Loading notes...