causal

Advances Come From Training Stages, Not Architecture

The rapid advancement in AI happens not in architecture but in the training stages—pre-training, mid-training, and post-training—where capability unlocks like supervised fine-tuning and RLHF (which turned GPT-3 into ChatGPT) come from the algorithmic side rather than architectural change.

causalpending

Speaker

Sebastian Raschka

Evidence Quote

“What was new was adding the supervised fine-tuning and the Reinforcement Learning with Human Feedback.”

Source

Ai Sota #2026— Lex Fridman Podcast
Created: 6/13/2026, 3:24:41 AM

Your notes

Loading notes...