factual

Complementary Tradeoffs of DL and Program Search

Deep learning (gradient descent) is compute-efficient but data-inefficient and limited to generalizing within its data distribution, while discrete program search (combinatorial search) is extremely data-efficient—learning a generalizable program from one or two examples—but compute-inefficient due to combinatorial explosion; every limitation of one is a strength of the other, so the path forward is to merge them.

factualpending

Speaker

François Chollet

Evidence Quote

“You can sort of see here how deep learning and discrete program search have very complementary strengths, and limitations as well... The path forward is going to be to merge the two.”

Source

Francois Chollet — Why the biggest AI models can't solve simple puzzles— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:26:16 AM

Your notes

Loading notes...