factual

Base models resist harmful requests before RLHF

Alignment researchers report that when red-teaming a base model that has not yet been RLHF'd—a raw next-token predictor—it is oddly hard to get it to help with things like making a bomb, with the model trying hard to refuse even before reward-based fine-tuning.

factualpending

Speaker

Dwarkesh Patel

Evidence Quote

“they will try to red team a base model. So it's not been RLHF'd. It's just "predict next token," the raw, crazy, shoggoth.”

Source

Joe Carlsmith — Preventing an AI takeover— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:37:01 AM

Your notes

Loading notes...