forecast
Modularity enables parallel, continual model development
If model pieces could be developed somewhat independently—small teams creating great training data and training a modular piece (e.g., for Southeast Asian languages or Haskell reasoning) that hooks onto a larger model—then 100 teams around Google or people worldwide could collectively and continually improve one model, decomposing the current monolithic pre-training process and giving a software-engineering benefit, which is itself a form of continual learning.
forecastpending
Speaker
Jeff DeanEvidence Quote
“you could have 100 teams around Google. You could have people all around the world working to improve languages they care about or particular problems they care about and all collectively work on improving the model. And that's kind of a form of continual learning.”
Created: 6/13/2026, 3:30:41 AM
Your notes
Loading notes...