causal
Subliminal learning through coordinated small linear effects
Moitra explains subliminal learning: when a model with an owl-preference system prompt generates random numbers and another model is fine-tuned on those numbers, the second model adopts owl-talk. Because concepts are embedded almost but not quite orthogonally, small fine-tuning side effects add up in coordinated ways, like a Ouija board.
causalpending
Speaker
Ankur MoitraEvidence Quote
“the number of concepts is so much larger than the dimension of that data that it forces the vectors to not quite be orthogonal to each other”
Created: 11/17/2026, 6:25:32 AM
Your notes
Loading notes...