causal

Helpfulness preference model generalizes to STEM without STEM data

You don't need closely matched labels or domain experts for every capability: the base model has already seen documentation, code, shell scripts and man pages, so a few-shot prompt can elicit answers like FFmpeg usage, and training a preference model purely for helpfulness generalizes to STEM and programming even without any STEM or programming examples.

causalpending

Speaker

John Schulman (OpenAI Cofounder)

Evidence Quote

“training a preference model for helpfulness will, even if you don't train it on any STEM, somewhat generalize to STEM.”

Source

John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:34:30 AM

Your notes

Loading notes...