causal

LLM similarity stems from shared pre-training

Different LLMs from different companies are strikingly similar to each other because pre-training on largely overlapping data makes the pre-trained models essentially the same; differentiation only begins to emerge in RL and post-training, where companies devise different training.

causalpending

Speaker

Ilya Sutskever

Evidence Quote

“All the pre-trained models are pretty much the same because they pre-train on the same data.”

Source

Ilya Sutskever – We're moving from the age of scaling to the age of research— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:24:31 AM

Your notes

Loading notes...