Pedro Domingos on Machine Learning and the Master Algorithm
What this covers
Pedro Domingos joins Russ Roberts to examine machine learning as a general-purpose technology that lets computers discover patterns and improve from data rather than following step-by-step human instructions. Domingos structures the field around five distinct schools—connectionists, evolutionaries, analogizers, symbolists, and Bayesians—each solving one piece of the learning puzzle. He argues these paradigms could unify into a single Master Algorithm, a grand unified theory analogous to physics unifying the forces of nature. The conversation ranges across the economic transformation machine learning enables, from matching buyers and sellers to navigating the long tail of infinite choice, and examines how causation differs from mere correlation—machine learning can discover causal relationships through experiments like A/B tests, not just spot associations in data.
Domingos takes an optimistic but contrarian stance on AI's dangers. Rather than fearing superintelligent machines with runaway goals, he argues the real problem is present-day stupidity: AI systems making error-prone decisions about credit, terrorism risk, and medical treatment without common sense or deep understanding. Humans set machines' objectives, so takeover scenarios are misconceived. The discussion touches medicine—how cancer's genetic variation demands personalized recommender systems rather than one-size-fits-all treatment—and work displacement, where Domingos contends that complementary goods and new job categories have historically absorbed workers freed by automation. He sketches consciousness as likely explicable within fifty years using new frameworks, though he brackets the subjective "hard problem" as beyond scientific reach.
Domingos argues that machine learning is a general-purpose technology for letting computers program themselves from data, that a single unifying 'Master Algorithm' is achievable, and that the real danger from AI is its stupidity, not its hypothetical superintelligence.
- The five paradigms of machine learning each solve one part of the problem, so a unified algorithm could combine their strengths
- Machine learning can discover causation through experiments (A/B tests), not just correlation
- AIs have no goals of their own; humans set their goals, so fears of machines taking over are misguided
Machine learning lets computers discover patterns and program themselves from data rather than receive step-by-step human instructions.
- Machine learning is computers programming themselves from data instead of being programmed step-by-step by humans; rather than telling computers exactly what to do, we let them look at the data we generate and figure out what to do.
“Machine learning is computers programming themselves instead of having to be programmed by us.”
- Even when humans don't think of themselves as learning, they are learning all the time at the lowest levels such as perception; these subconscious capabilities that people take for granted are actually the hardest things, which is why computers struggled with vision and speech recognition until machine learning helped.
“even when we natural human beings don't think of ourselves as learning, we are learning all the time. I'm talking about learning even at the lowest levels, like perception.”
- Big data and machine learning are symbiotic: as the amount of available data grows exponentially, machine learning grows exponentially more powerful by piggy-backing on it, and conversely machine learning is what makes big data useful since the reason to capture and store data is to learn models from it.
“So machine learning is what makes big data useful. And big data is what makes machine learning powerful.”
- Machine learning is a subfield of artificial intelligence; AI is the subset of computer science dealing with the hardest problems requiring intelligence (problem solving, vision, language, common sense), and of all these capabilities the ability to learn is the most important, since an intelligent agent that couldn't learn would immediately fall behind and never catch up.
“machine learning is a subfield of artificial intelligence. Artificial intelligence is the subset of computer science that deals with, in some sense, the hardest problems.”
- Machine learning differs from traditional statistical methods like linear regression in two big ways: it embraces the full gamut of nonlinear models rather than only straight-line relationships, and it learns from data on all individuals rather than small aggregate data that omits important details.
“traditional statistical methods are very, very limited. They are things like linear regression. The only relationships you can find are the ones that follow a straight line.”
Five distinct machine learning tribes each handle different learning problems and must merge into one grand unified algorithm.
- No single one of the five paradigms can solve the whole learning problem because each handles only one part well; what is needed is a unification of all five into one algorithm—a grand unified theory of learning analogous to physics unifying the forces of nature—that combines all their capabilities and could in principle learn anything given enough data.
“none of these [?] 5 paradigms is going to solve the whole learning problem... what you need to solve all of them is the unification of the five.”
- The Master Algorithm relates to learning the way a microprocessor relates to integrated circuits: a microprocessor is never the best way to do any single thing—an application-specific circuit always beats it—yet it is used for 99.9% of things because it is one programmable thing everyone can have; similarly the Master Algorithm won't be optimal for everything but will do about 80% of the job universally.
“a microprocessor is not the best way to do anything... And yet, microprocessors are what we use to do 99.9% of things. Precisely because it's just one thing”
- The history of machine learning shows paradigms rising and falling by decade: connectionism dominated in the 1950s-60s before symbolists killed it, revived in the 1980s, faded as Bayesians then analogizers ruled, and is now ascendant again as deep learning, illustrating that today's dominant paradigm is not necessarily permanent.
“in any given decade there is one of these paradigms that is sweeping all before it; but then come the next decade, the paradigm has fallen behind and another one is in the lead.”
- There are five tribes of machine learning: connectionists (reverse-engineering the brain via neural networks and connection strengths), evolutionaries (evolving programs the way nature evolves creatures), analogizers (reasoning by analogy / similarity), symbolists (formulating induction as the inverse of deduction), and Bayesians (quantifying uncertainty over hypotheses with probability and updating with evidence).
“The connectionists are the people whose agenda is to do machine learning inspired by how the brain works.”
Machine learning enables personalized solutions by learning from individual data rather than aggregate patterns.
- Machine learning is heading toward a single big model that sees all aspects of a person—a 360-degree view—rather than fragmented models from different companies; this model would act as a faithful lifelong companion across college, jobs, and relationships, but it should not be owned by Google or Apple because that would give them too much power, so it must be built to be used only on the individual's behalf.
“all of it goes together into one big model that sees all aspects of you. Sees you--it's like a 360 degree view of you.”
- Online dating sites use machine learning to propose potential dates, and since about a third of all marriages now start on the Internet, there are children alive today who would not have been born without machine learning.
“these days, a third of all marriages start on the Internet, so they are actually children alive today that wouldn't have been born if not for machine learning”
- The cure for medical reversal and treatments deployed on little evidence is to be more evidence-based: instead of basing decisions on one small study, machine learning lets us continuously gather feedback from treatment results as they roll out, dropping things that fail at scale and identifying the specific 5% of patients for whom a treatment actually works.
“Where medicine is going is that we are continuously getting the feedback from the results of treatments. And, based on those results, certain things will rise or sink.”
- Cancer is hard to cure because it is not a single disease: everybody's cancer is different and even the same person's cancer differs three months later because it mutated, so the best solution is a machine-learning algorithm that predicts the best drug given the patient's genome, medical history, and the cancer's mutations.
“cancer is hard to cure, because it's not a single disease. Everybody's cancer is different. And even the same person's cancer three months later because it mutated.”
- Whether two people will hit it off is not just a function of who they are but of how their interaction went; the same two people could fall in love or annoy each other depending on the conversation's track, so a future dating system would run a thousand simulated dates between a pair and recommend the matches with the highest success rate.
“whether or not two people will hit it off is actually not just a function of who they are. It's a function of how things went.”
Machine learning augments human capability and complements human weakness rather than wholly replacing human workers or judgment.
- Even 20 years after Deep Blue beat Kasparov, the world's best chess players are not computers but teams of humans and computers, because humans and computers have complementary strengths and weaknesses; a human with a computer can beat a computer alone, and the same complementarity will hold for many jobs that ML augments rather than replaces.
“Even now, 20 years after Deep Blue beat Kasparov, the world's best chess players are not computers. They are teams of humans and computers.”
- Fears that computers will get too smart and take over are misguided because they confuse AIs with people; real AIs have no goals of their own since we set their goals, so the actual danger is that today's computers do damage by being too stupid—making error-prone decisions about credit, terrorism risk, and dates—and the cure is to make them more intelligent so they have common sense and understand what we tell them.
“People worry that computers will get too smart and take over the world. But the real problem is they are too stupid, and they've already taken over the world.”
- Machine learning algorithms are typically better than human doctors at diagnosing various things, such as deciding from an x-ray whether there is a tumor in someone's chest.
“Medical diagnosis, machine learning algorithms are typically better than human doctors at diagnosing various things, like for example deciding by looking at an x-ray whether there is a tumor in somebody's chest or not.”
- Machine-learning programs, no matter how much they learn, do not have goals of their own; humans program them at a higher level by specifying goals and letting the machine figure out how to achieve them, so it remains our job to set their goals.
“These machine-learning programs, no matter how much they learn, they do not have goals of their own. It is still our job to tell them what their goals should be.”
Machine learning makes economic markets more efficient by matching buyers and sellers and reducing transaction costs.
- Recommender systems predict what you want to buy by looking at what you and other similar people have bought or clicked on.
“from the things you've bought or clicked on or that other people have bought or clicked on will try to predict what it is that you want to buy and then recommend that to you”
- Machine learning makes the economic abstraction that all buyers can transact with all sellers closer to reality by matching buyers and sellers and reducing transaction and search costs; the online ad auction, run billions of times per day, exemplifies ML enabling the ideal model of how economics works.
“the ideal model of how economics works in some sense is actually being enabled by machine learning literally at the rate of billions of auctions per day for just the case of ads”
- The Internet creates a world of nearly infinite choice, which causes a problem of too many choices; machine learning is the next logical step that lets you make those choices without examining every item, picking out what you would have chosen if you could survey everything, thereby making the long tail navigable.
“The Internet creates all these choices because it puts everybody in contact with everybody. But then what machine learning does is it actually lets you make those choices without having to look at every item yourself.”
Machine learning can discover causal relationships through experimentation and intervention, not merely correlation.
- Causality, at the end of the day, just means being able to predict the effects of your actions; a notion of causality that does not involve predicting the effects of your actions would be indistinguishable from correlation, and machine learning can be used to predict the effects of your actions.
“Causality is being able to predict the effects of your actions.”
- It is a common misconception that machine learning can only discover correlation; ML can and does discover causation by performing experiments such as A/B tests and randomized trials, and can go further than asking whether a drug works to identify exactly who it works for by examining all the other variables involved.
“This is actually a very common misconception about machine learning, is that it can only discover correlations. Machine learning can and does and has discovered causation, not just correlation.”
Automation displaces jobs but creates new ones through lower costs and emerging categories, and democratic pressure will ensure redistribution.
- It is always easier to see jobs lost to automation than jobs created, but cheaper goods free up disposable income to demand complementary goods and whole new job categories emerge; just as 19th-century American farming jobs disappeared without leaving 95% unemployed, people now do once-unimaginable jobs like app designer.
“In the 19th century, most Americans were farmers; and then all those jobs disappeared. But it's not like 95% of Americans aren't employed. They are just doing things that were unimaginable back then, like App Designer.”
- If automation eventually pushes unemployment toward 50%, then in a democracy where everyone votes, the roughly 95% who would be on the receiving end will vote for generous benefits or wealth redistribution such as a universal basic income; this is hard to prevent because, unlike measures that fly under the radar, such a shift would be highly visible.
“Once the unemployment rate reaches 50%, everybody is going to vote for very generous unemployment benefits--or for redistributing the wealth. In a democracy, everybody has a vote.”
Consciousness will become scientifically understood within fifty years but its subjective hard problem may remain beyond objective science.
- One of the most intriguing approaches to artificial intelligence is building a robot baby: create something with the same capabilities (not necessarily at the neuron level) that a newborn's mind has, give it the same experience a newborn gets, and two or three years later you will have solved the problem—it is the only path to AI we know is guaranteed to work because a human child is the existence proof of intelligence.
“the only one that we know is guaranteed to work, right? Is to actually have a real being in the real world learning from experience in the same way that a baby does.”
- Robots would need the equivalent of emotions to learn, because emotions are how evolution programmed humans to choose the right actions; at a functional level we already know how to put emotions into a robot via the objective or utility function, and the input-output behavior could be indistinguishable from a robot having emotions, though whether it truly has them is something we may never know.
“Emotions are the way evolution in some sense programmed you to do the right things and not the wrong ones... And we know how to do that in a robot. The technical term for that is the objective function”
- Within roughly 50 years we will likely have a satisfying model of consciousness using new concepts that explains its psychological and neural correlates, much as life ceased to seem mysterious after DNA; but because consciousness is a subjective experience with no possible objective test, the 'hard problem' of consciousness will likely remain beyond the scope of science.
“if you fast forward to 50 years from now, we will probably have a very good and very satisfying model of consciousness.”
- Reasoning by analogy, where we solve a new problem by retrieving the most similar past situations and extrapolating their solutions, is a powerful and natural approach, but it can fail in two ways: picking the wrong analogous case, or picking the right case but transferring the wrong features from it.
“Whenever you are doing an analogy between two things, some things are similar, some things aren't. And if you transfer the wrong things, it could also fall flat that way.”
- The great majority of AI researchers do not believe in the Singularity and think it a rather silly idea, because we never understood the world more than a little bit anyway; computers already do things people don't understand without the world collapsing, precisely because systems are designed to keep an understandable interface to us since they exist to serve us.
“the great majority of AI researchers do not believe in the Singularity. In fact, they think it's kind of a ridiculous idea.”
- The goal in machine learning is not to predict perfectly but to predict well enough to be useful; this differs from the laws of physics, and many of the most successful ML applications started from a very low baseline, such as stock prediction where being consistently 51% accurate is enough to make you very rich.
“the goal in machine learning is not to predict perfectly. It's just to predict well enough to be useful.”