What this covers
Tyler Cowen speaks with Daniel Kahneman about three interconnected claims that have come to preoccupy the psychologist in recent years. The first concerns noise vs bias: an insurance firm's study revealed that professional underwriters differed by 50% in their risk assessments of identical cases, far exceeding the 10% variability executives had anticipated. Kahneman argues that noise—random, unsystematic disagreement between equally qualified judges—has been vastly underestimated as a source of error, while the research literature has fixated almost exclusively on bias. The second centers on intuitive expertise. Kahneman draws on Gary Klein's work to specify the narrow conditions under which intuition proves reliable: a regular environment with detectable patterns, extensive experience, and rapid feedback. Outside those bounds, gut feeling should be delayed until facts are assembled and examined independently. The third thread concerns memory. Kahneman proposes that people optimize for recalled satisfaction rather than moment-to-moment happiness, since only memory persists; the experiencing self vs remembering self pursue different goals, and the remembering self has outsized influence on life satisfaction.
The conversation moves fluidly across application domains. Kahneman discusses Phil Tetlock's finding that ordinary people trained in careful reasoning outforecast CIA analysts; the proper conditions for overriding algorithmic judgments; why wisdom of crowds only reduces error when individual biases are uncorrelated; how loss aversion vanishes in routine transactions but governs novices; the replication crisis and cognitive behavioral therapy's empirical standing; and whether machines will eventually outpace humans in medicine and chess. He also addresses whether artificial systems can develop genuine common sense, the evolutionary roots of duration insensitivity in pain perception, and how optimism, though individually risky, fuels capitalism. Throughout, Kahneman distinguishes sharply between sunk cost as a decision error and the legitimate attachment that comes from familiarity and trust.
Kahneman argues that noise (random variability in judgment) is a vastly underestimated source of error relative to bias, that intuition should generally be delayed except under narrow conditions of true expertise, and that human well-being is governed more by remembered satisfaction than by experienced happiness.
- Underwriters at an insurance firm differed in their risk assessments by 50% when everyone expected ~10%, revealing massive hidden noise
- Noise and bias are independent error sources requiring different remedies, but the literature has focused almost exclusively on bias
- Memory, not momentary experience, is what people retain and optimize for, so they maximize life-satisfaction rather than experienced happiness
Noise is a vast, underestimated source of error that exceeds bias in prevalence and harm.
- In an insurance experiment where 50 specialists priced realistic, routine fraud-risk cases, the variability between two random underwriters averaged 50% (relative to the mean premium), versus the ~10% that executives expected — meaning the underwriters were effectively wasting their time assessing risk, demonstrating that noise (random variability between supposedly interchangeable judges) is far larger than anyone imagines.
“what we found was 50 percent, 5–0, which, by the way, means that those underwriters were absolutely wasting their time, in the sense of assessing risk.”
- Noise and bias are independent sources of error, so reducing either improves accuracy, but they require different reduction procedures; while it is hard to say which is more prevalent, bias has clearly been overestimated at the expense of noise, since virtually all the literature and public conversation focuses on biases while noise is extremely important and prevalent.
“noise and bias are independent sources of error, so that reducing either of them improves overall accuracy... the procedures by which you would reduce bias and reduce noise are not the same.”
- Wherever people exercise judgment there is noise, and there is more noise than people expect and more than they can imagine, because it is hard to imagine that others hold very different opinions when you believe your own opinion is correct — which is why nobody in the insurance organization had any idea the variability existed.
“The thing that was the most striking was that nobody in the organization had any idea that this was going on. It took people completely by surprise.”
- Failing to gather adequate information when it is available, or being swayed by the first thing that comes to mind, is a common stupid behavior that is neither exactly a bias nor low intelligence per se, and it contributes more to noise than to bias.
“When people collect too little information or are swayed by the first thing that comes to mind, you get noise rather than bias.”
- The noise book will argue that noise is an underestimated problem and that there is something deep about two ways of thinking — statistical versus causal: noise is inherently a statistical way of looking at things while bias is inherently causal, and the interplay of these forms of thinking is central.
“Noise is clearly a statistical way of looking at things, and bias is inherently causal — so the interplay of those forms of thinking.”
Genuine intuitive expertise requires regular world, extensive experience, and rapid feedback; most people lack these conditions.
- CEOs do not encounter many problems where they have genuine intuitive expertise because they have not had the repeated, regular-feedback opportunities to acquire it, unlike athletes, chess masters, and firefighters — so CEOs should slow down rather than trust their intuition.
“I don’t think CEOs encounter many problems where they have intuitive expertise. They haven’t had the opportunity to acquire it, so they better slow down.”
- Genuine (Klein-type) intuitive expertise develops only when three conditions hold: a regular world with recognizable regularities, a lot of experience, and rapid unequivocal feedback; under those conditions intuition will be reliable (as for chess players or spouses reading each other's emotions), but absent them we cannot trust people who claim to be experts.
“You’re going to have Gary Klein–type intuition, expert intuition, if you have a regular world... There are regularities that you can pick up. And if you have a lot of experience and if the feedback is rapid and unequivocal.”
- Because reputation is the best available measure for choosing a doctor, the main real index for a surgeon is the number of times they have performed the specific operation in question, since people genuinely get better with practice and that practice is fairly specific to each operation.
“If it’s a surgeon that I’m looking for, then there are real indices, the main one being the number of times he’s performed the operation in question.”
Pooling independent judgments and using validated algorithms reduces noise more reliably than individual decision-making.
- To improve judgment quality, delay intuition until the facts are at hand and look at the dimensions of a problem separately and independently; because intuition forms too quickly, you need special procedures to control it, breaking decisions into stages and not acting until information is available — except in rare cases of genuine intuitive expertise.
“I think delaying intuition is a very good idea. Delaying intuition until the facts are in, at hand, and looking at dimensions of the problem separately and independently is a better use of information.”
- Google requires a minimum of four individuals to make independent assessments of each job candidate, which reduces the standard deviation of error by at least a factor of two — a concrete organizational example of pooling independent judgments to reduce noise.
“Google, for example, when it hires people, they have a minimum of four individuals making independent assessments of each candidate. And that reduces the standard deviation of error at least by a factor of two.”
- Pooling opinions (wisdom of crowds) only reduces error when individual errors are independent; when everyone is inclined to make the same mistake (a shared bias), pooling worsens the outcome and merely raises everyone's confidence in their biased views, so the wisdom of crowds works only under quite specified conditions.
“The wisdom of the crowds will work, and pooling opinions will work when errors are independent”
- Prediction markets reduce noise because averaging does that, but they cannot reduce a general bias unless the unbiased people also know they are unbiased and can therefore invest more to move the price toward the correct answer; without that asymmetry of knowledge, a bias will persist.
“The wisdom of the crowds will work, and pooling opinions will work when errors are independent”
- Once a validated machine or formula is making decisions, the conditions under which it is worth overriding it are narrow and well understood — a mere feeling that the machine is wrong is not enough; legitimate override requires decisive new information (e.g., a loan applicant just arrested for fraud), and in many experiments people who override formulas do worse than if they hadn't.
“when people override formulas, by and large, they do worse than if they hadn’t.”
People systematically misjudge their own judgment quality and cannot eliminate biases through single decisions.
- Knowing about a bias does not let you simply decide once and for all not to be subject to it; you can sometimes talk yourself out of a specific bias with a lot of repeated effort, but it cannot be eliminated by a single decision.
“It’s not that you can decide once and for all, “I will not be subject to that bias.” Doesn’t work that way.”
- Our basic ideas about what is computationally difficult versus easy have undergone revolutionary changes: we used to think perception would be easy and thinking difficult, then it turned out thinking was relatively easy and perception difficult, and now with new ways of handling perceptual problems, thinking is difficult again.
“We used to think that perception would be easy, and thinking would be difficult. It turns out that thinking was relatively easy and perception was difficult. Now, there are ways of handling perceptual problems, and so thinking is difficult again.”
- Easier access to information makes it easier to be rational in certain domains where you can look things up, but human nature and the underlying biases have not changed over the past 30 years; moreover, the same channels that spread information also spread misinformation, so the once-clear benefit of availability now carries a heavy cost.
“Human nature hasn’t changed. In certain domains, it’s much easier to be rational when you can look things up... But whether it has changed anything significant, I doubt it.”
- Trying to persuade another person to listen to your opinion with an open mind and overcome their own biases is essentially a game one primarily plays with one's spouse, and by and large it doesn't work.
“It’s a game one primarily plays with one’s spouse, and it doesn’t work, I think, by and large.”
Memory rather than experienced happiness drives life evaluation because it persists while momentary experience vanishes.
- The duration of a painful experience matters very little for how we evaluate it; what matters is intensity, because intensity measures the severity of threat, which has evolutionary value, whereas duration is evolutionarily insignificant.
“the duration of pain is really not very important. What’s important is the intensity because the intensity is a measure of the severity of threat.”
- Happiness feels good only in the moment and the reality of life vanishes instantly, but memories stay with us — so memory has a disproportionate weight in how we value our lives because it is the only thing we get to keep.
“What you’re left with are your memories. And that’s a very striking thing — that memories stay with you, and the reality of life is gone in an instant. So memory has a disproportionate weight because it’s with us. It’s the only thing we get to keep.”
- People do not maximize experienced happiness; they want to maximize satisfaction with themselves and with their lives, which leads in completely different directions than the maximization of momentary happiness — a divergence that led Kahneman to leave the field of happiness research.
“I don’t think that people maximize happiness in that sense... They actually want to maximize their satisfaction with themselves and with their lives. And that leads in completely different directions than the maximization of happiness.”
- If you want good memories, good endings are really important — particularly in the context of goal striving, where whether you achieve or fail to achieve a goal colors the whole experience of trying to reach it.
“if you want good memories, good endings are really important... whether you achieve a goal or don’t achieve a goal colors the whole experience of trying to get it.”
Many emotions and preferences reflect moral training or evolutionary value, not cognitive biases or errors.
- Regret and guilt are counterfactual emotions driven by what could have happened but didn't; some, like regret, seem superfluous and people would by and large be better off without it — yet regret is what happens the next morning, and without that anticipated cost there is no telling what we might do.
“Some of these emotions seem to be completely superfluous, like regret. And I think people, by and large, would be better off without regret. But notice what regret is. Regret is what happens the next morning. And if we didn’t have it, then who knows what we might do.”
- Shame is probably a force that induces better behavior in many people who would not otherwise be controlled, so while it is painful to the people who feel it, it may be useful to others who would be affected by their bad behavior.
“It’s probably a force that induces better behavior in lots of people who would not be controlled in other ways... It’s painful to the people who feel it, and it might be useful to others who might be affected by bad behavior.”
- Feelings that certain transactions (e.g., markets in kidneys) are repugnant are not biases but moral emotions — indignation and moral disgust — that we are trained or socialized to feel without being able to explain why, as shown by Paul Rozin's experiment where people refuse to drink orange juice they themselves labeled 'cyanide.'
“there are things that we’re trained or socialized to find disgusting, to find repugnant... There is such a thing as moral emotion. There is such a thing as indignation, as moral disgust.”
- Rooting for your home team or feeling pride when a stranger on your street wins a prize is not a bias but an emotion over which you have little control; the tendency to identify with what's around us and things we're connected to is very powerful, and labeling any such emotion a bias is tendentious.
“There are emotions over which you have very little control. It’s a fact that you feel pride when your team wins... That tendency to identify with what’s around us, and with things that we are connected to, is very powerful.”
Current frameworks and findings will become obsolete within decades as entirely new thinking emerges.
- Noise is useful in exactly one condition: when there is a selection process, because evolution works on random variation followed by selection; but when there is no selection, noise is just a cost.
“If there is a selection process, evolution works on noise. You have random variation and then selection. But when there is no selection, noise is just a cost.”
- The belief that we will always need human beings working with machines is an illusion; in chess, human-plus-program teams were briefly stronger than programs alone but are no longer, and the same pattern is emerging in dermatology where diagnosis is now better done by programs — and human intervention will more often introduce mistakes than correct them, so a well-running program should be left alone.
“That we will need human beings is, I think, an illusion. Take chess for example. Kasparov was beaten 20 years ago... The programs do not need the grand masters.”
- Kahneman's research adds nothing to how students should choose colleges or careers; the sensible methods are well known — collect a lot of information and ask what the student really wants and where they will fit — and if a student has an inarticulate gut feeling, the right response is to probe where that feeling comes from rather than defer to it or to an algorithm.
“My research adds absolutely nothing to this. There are sensible ways of choosing colleges... You have to collect a lot of information, and you have to ask yourself what the student really wants and where he or she will really fit.”
- A new CEO has one big advantage: no sunk costs with respect to the poor ideas the exiting CEO had and couldn't let go of.
“when you get a new CEO in place in organizations, the new CEO has one big advantage. He’s got no sunk costs with respect to poor ideas that the exiting CEO had and couldn’t let go of.”
- Essay grading contains so much noise that it is easy to imagine a machine-learning program looking at various indices that would grade better than hurried and tired professors.
“there is so much noise in essay grading that it’s quite easy to imagine a program that would look at various indices and that would do better than hurried and tired professors.”
- Exaggerating one's odds of success is useful to the individual because it makes them more appealing, attracts resources, and induces risk-taking — even though the expected utility of those risks is probably moderately negative for them; but for society as a whole, having many optimists taking risks is what drives economic progress, making optimism the engine of capitalism.
“for society as a whole to have a lot of optimists taking risks — that’s what makes for economic progress, so I call that the engine of capitalism, really, that sort of optimism”
- Researchers tend to persist with projects too long because of sunk costs; in innovative research one should recognize when something isn't working and move on — sunk cost is properly defined as placing a different value on a move or investment because of an investment already made, distinct from the legitimate growing attachment to things one trusts.
“I think sunk cost is really the enemy when you’re doing research, innovative research. You’re to recognize that something isn’t working and just move on.”
- Everything Kahneman currently believes will become irrelevant within a few decades, not through correction of details but through entirely different frameworks and ways of thinking emerging — which is what happens to all ideas and frameworks over time.
“This is what happens to ideas or to frameworks. They, at some point, become irrelevant. And I know that this is going to happen to everything that I believe. Give it a few decades — it’s going to be irrelevant.”
- The five-factor personality model dominates personality psychology, despite being about 20 years old and somewhat boring, precisely because it works — having replaced more exciting but complicated mechanisms with something that seems to function.
“This five-factor thing — it’s about 20 years old, and it dominates personality psychology because it works.”
- There are genuine cultural differences in attitudes toward optimism and risk-taking: in quite a few European countries optimism is considered rather foolish and childish, whereas in the United States optimism is clearly a desirable trait, with similar cross-cultural variation in whether risk-taking is seen as good or bad.
“In quite a few European countries, optimism is considered rather foolish. It’s for children. And in the United States, optimism is clearly a desirable trait.”
- The continuous laughter and humor in Kahneman and Tversky's joint work was important to its nature because they were examining their own thinking, finding stupid things in it, and finding that delightful — a topic choice that lent itself to things virtually impossible in other fields.
“we were examining our own thinking, and finding stupid things in our own thinking, and finding that delightful and very funny. So we were very lucky in our choice of topic in many ways.”
- Happiness-promotion interventions are very difficult to evaluate because people who know they are being subjected to an intervention cannot answer questions about their happiness honestly even if they try; a better test would be to ask the person's friends whether they have become happier, which hasn't been done.
“it’s very difficult to conduct evaluations on those things because people who know they’re being subjected to interventions cannot really answer those questions honestly, even if they try.”
- Among therapies, cognitive behavioral therapy clearly works and is evidence-based and supported time and again because it is a technique, whereas other therapies' effectiveness primarily depends on the personality of the therapist and the interaction between therapist and patient — and the rest is a lot of bias.
“there is one line of therapy that clearly works, and it’s evidence based, and it’s supported time and again... it’s cognitive behavioral therapy. That works, and we know it does.”
- The serious psychological question about AI is not the engineering of the last 1% but whether computers can simulate common sense and genuinely understand what they are talking about; lacking sense, whims, perceptual apparatus, and the ability to cause things by acting on the world, programs cannot be exactly like us, and nobody today claims even the most sophisticated programs have that sense of understanding.
“The questions that are of interest as a psychologist is, when can you simulate common sense? There is the really serious question that people raise about computers, whether they know what they’re talking about, whether they understand what they’re talking about.”
- ADHD is not a bias and not noise but an attention deficit — a difficulty controlling attention, focusing on what one wants to focus on, and staying focused — demonstrating that bias and noise do not cover the universe of error categories.
“I think it’s an attention deficit. It means that people have difficulty controlling their attention... That’s neither bias nor noise. Bias and noise do not cover the universe. There are other categories.”
Ordinary people can forecast better than experts by adopting multiple viewpoints and treating problems as category instances.
- Tetlock's research shows beyond doubt that ordinary people, thinking carefully without any training, can make medium-term forecasts (up to about six months) that beat CIA analysts; they succeed by viewing the problem as an instance of a category, then switching to look at it from the inside and adopting multiple points of view — and once selected as the top 2%, superforecasters do not regress to the mean.
“you can help people thinking carefully without any training, who do better than CIA analysts.”
Replication crisis has substantially improved psychology and other sciences by exposing unreliable findings.
- The replication crisis began in medicine with claims that most published research is false, reached psychology very soon after, and within a decade has transformed many areas of psychology into a clearly better science than it was ten years ago — despite aggressors and defenders often behaving badly.
“within a decade, psychology has changed. Many areas of psychology have changed, and it clearly is a better science than it was 10 years ago because of the replication crisis.”
- Although some results published in Thinking, Fast and Slow are not reliable and Kahneman wishes he hadn't published them, the core idea of two systems is immune to replication concerns because it is anchored in basic experience: getting 2 plus 2 happens automatically and quickly with no control, while getting 17 by 24 demands slow effortful processing — these are extremes on at least a continuum worth exploring.
“The idea of two systems is really anchored in a basic sort of fact of experience, that the process by which you get 2 plus 2 is fundamentally different from the way that you get 17 by 24... That’s immune to replication.”
Loss aversion applies only in attachment-driven contexts, not in routine expert transactions where rational behavior emerges.
- Loss aversion plays a large role for novices who get attached to things and don't want to sell them, but it largely disappears in routine transactions — when buying shoes, neither buyer nor seller feels loss aversion — so there is a whole domain in which loss aversion simply doesn't apply, consistent with research showing experts behave more rationally.
“In routine transactions, when I go and I spend some money to get shoes, I feel no loss aversion for the money. And certainly, the person who sells me the shoes feels no loss aversion for the shoes.”
Rational-agent consistency model is infeasible and incomplete; better normative theory rests on reasonableness and human context-dependence.
- The rational-agent model treats consistency of beliefs and preferences as the single normative principle, but consistency is both infeasible for finite minds and an odd sole criterion; a better normative theory would be built on reasonableness and would take into account human nature, which is inherently context-dependent and not consistent.
“As a normative principle — that consistency is the only normative principle — that strikes me as pretty odd... the idea of consistency — it’s completely infeasible for a finite mind, and we have finite minds.”