transfer learning
The goal of having an AI apply knowledge learned on one task to new, different tasks (e.g., from chess to checkers, or from CT scans to x-rays); equivalent to what humans simply call learning
As defined in Melanie Mitchell on Artificial Intelligence
Other definitions
The idea that skills practiced in one context, including games, carry over to different tasks that share underlying cognitive patterns.
Building a Modern Business
Where it comes up · 6
- Building a Modern Business2 claims
- Melanie Mitchell on Artificial Intelligence2 claims
- Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold2 claims
- How to Build a Great Product1 claim
- Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold1 claim
- Fully autonomous robots are much closer than you think – Sergey Levine1 claim
Most important claims · 9 of 9Search all claims for “transfer learning” →
- A trained classifier can recognize new instances within the statistical distribution of its training data (a new photo of a dog), but cannot recognize out-of-distribution cases (a cartoon or painting of a dog) or transfer knowledge to a genuinely new domain—so what AI does is not learning in the human sense, since human learning implies using knowledge in novel situations.Melanie Mitchell on Artificial Intelligence
- Brazilian players became skilled because a culture of small-space pickup soccer gave them far more ball contact than the formal British system did, showing that a different game can still teach the same underlying skill.Building a Modern Business
- Computer vision algorithms trained on one set of images often work surprisingly well on other problems, sometimes not even vision-related, suggesting a convergence with how the brain generalizes its learned seeing skills.How to Build a Great Product
- Robotics is harder because it is a data-poor regime, but treating any input—an action, a word, an image pixel—as just a token (as in Gato) lets large models transfer to robotics, and true multimodality produces cross-modal benefit where understanding video improves language, so general multimodal systems will ultimately be more capable than single-modality ones despite being harder to train initially.Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold
- AI's accomplishments—beating top chess and Go players, speech and handwriting recognition, machine translation, medical image analysis—are each narrow tasks requiring a system built largely from scratch; none generalizes, so a system that masters Go cannot even play a variation of Go, let alone checkers.Melanie Mitchell on Artificial Intelligence
- Robotics remains a valuable research domain despite being data-poor because it forces innovation in sampling efficiency, data efficiency, transfer learning, and sim-to-real transfer that benefits all AI systems.Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold
- Large multimodal models can treat different modalities as tokens (words, pixels, actions, etc.), enabling transfer across domains; training such systems is harder than pure language models, but the resulting multimodality produces better performance across all modalities as they learn from each other's structure.Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold
- Predicting video has not produced deep world-understanding the way predicting text has, because a visual scene contains unbounded detail (clouds, water molecules, pedestrians) with no inherent prioritization, whereas text is already abstracted into the bits humans care about, providing good representations focused on what matters.Fully autonomous robots are much closer than you think – Sergey Levine
- Video games can build real-world skills through transfer learning, because the cognitive patterns trained in play carry over into other domains even when the game looks different from the target skill.Building a Modern Business