verifiable domains
Problem areas like math and code where answers can be objectively checked, reducing the risk of embedded cultural bias when training or distilling models.
As defined in Mark Zuckerberg — AI will write most Meta code in 18 months
Where it comes up · 2
- Ben Thompson on Big Tech, China, and the AI Boom Running Out of Money - [Invest Like the Best, EP.487]3 claims
- Mark Zuckerberg — AI will write most Meta code in 18 months1 claim
Most important claims · 4 of 4Search all claims for “verifiable domains” →
- AI's strong results in chess, Go, coding, and math come from bounded, verifiable domains, and there is not yet compelling evidence that this translates to unverifiable or long-verification-loop domains, which make up a large share of economic activity.Ben Thompson on Big Tech, China, and the AI Boom Running Out of Money - [Invest Like the Best, EP.487]
- Google and Meta have a unique advantage in using AI for advertising because ad A/B testing at global scale provides a verification loop in which human clicks and purchases validate generated images and text, and predictive models can lift relevance by a few percentage points worth billions of dollars.Ben Thompson on Big Tech, China, and the AI Boom Running Out of Money - [Invest Like the Best, EP.487]
- Models trained on internet text learn the end products of human thought but not the underlying traces of reasoning, emotion, and process, so capturing those traces, possibly through devices like Neuralink, could be the real source of capability gains.Ben Thompson on Big Tech, China, and the AI Boom Running Out of Money - [Invest Like the Best, EP.487]
- Distilling language models is fraught because of embedded values, but for reasoning models limited to verifiable domains you can distill quite securely by combining code cleanliness and security filters, tools like Llama Guard and Code Shield, and extensive red teaming to check the model isn't doing anything unwanted after distillation.Mark Zuckerberg — AI will write most Meta code in 18 months