factual

DeepSeek's research taste shows hardware-algorithm co-design

DeepSeek's good research taste manifests as designing models perfectly to their hardware constraints: they used MLA to trade flops for memory bandwidth on flop-rich H800s, then moved to more memory-bandwidth-oriented NSA as export controls reduced their chips, and improved their MoE load balancing from auxiliary losses to cleaner bias terms—iteratively co-designing algorithm and hardware in a simple, elegant way backed by great engineering.

factualpending

Speaker

Sholto Douglas

Evidence Quote

“the models give this sense of being perfectly designed up to their constraints. You can really very clearly see what constraints they're thinking about as they're iteratively solving these problems.”

Source

Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:28:51 AM

Your notes

Loading notes...