Austin Frakt on Medicaid and the Oregon Medicaid Study
What this covers
Austin Frakt and Russ Roberts examine the Oregon Medicaid study, a randomized controlled trial created when Oregon allocated Medicaid expansion slots by lottery. The conversation centers on how to interpret the study's mixed results: strong, statistically significant gains in financial protection and mental health, but null findings on physical health measures like blood pressure and cholesterol. Frakt's core argument is that the study's non-significant physical health results should not be read as evidence that Medicaid has no effect; instead, they are genuinely uninformative because the study was statistically underpowered to detect realistic effects. The discussion then broadens to how people actually use empirical evidence in policy debates—and often do not, allowing confirmation bias to dominate.
The conversation covers several distinct territories. On the study itself, Frakt explains why the physical health results remain silent: the trial would have needed to detect enormous effect sizes (such as an 80% reduction in elevated blood sugar) to reach statistical significance, making the observed null results consistent with a plausible small benefit rather than proof of zero effect. The financial and mental health findings stand on firmer ground—catastrophic medical debt fell sharply, and depression scores improved substantially. Frakt also clarifies that Medicaid is not a single program but two layers: a federally mandated core for categorically eligible poor and a state-optional expansion for the merely poor, a distinction that shapes which populations benefit most. On methodology, both speakers reflect on weight of evidence across multiple studies rather than betting on any single result, and on how price incentives drive behavior in health care—with the earlier RAND Corporation Health Insurance Experiment showing that lower prices increase use without necessarily improving health for most people. The episode closes with pessimism about whether legislators will let the evidence reshape their priors, or merely cite whichever results align with positions they held beforehand.
Frakt argues the Oregon Medicaid study was statistically underpowered to detect physical health effects, so it should be read as showing clear financial-protection and mental-health benefits while remaining genuinely uninformative—not negative—on physical health outcomes.
- The non-significant physical health results had error bars too wide to distinguish a plausible small Medicaid effect from zero
- The study found robust, statistically significant gains in financial protection, access, and mental health
- Single studies should not be definitive; a body of work using varied methods matters more, but methodological strength like randomization weighs heavily
The Oregon study was too small to detect realistic Medicaid effects on physical health outcomes.
- The latest Oregon study found roughly a dozen physical health measures—blood pressure, cholesterol, blood sugar control, and related medications—none of which were statistically significant, creating the appearance that Medicaid has no effect on physical health, which is the core of the controversy.
“when you look at the table in the paper there's maybe a dozen or so results that are staring at you, all about physical health. In one table. And none of them are statistically significant. So it appears as if the study is saying that Medicaid is not able to have an effect, across all of these health measures.”
- The Oregon study was statistically underpowered to detect realistic Medicaid effects on physical health: the effect sizes it could have detected were enormous (e.g., it would have needed an 80% reduction in elevated blood sugar versus the observed 20% point estimate), so the non-significant results are genuinely uninformative rather than evidence of zero effect.
“when you look at the table in the paper there's maybe a dozen or so results that are staring at you, all about physical health. In one table. And none of them are statistically significant. So it appears as if the study is saying that Medicaid is not able to have an effect, across all of these health measures.”
- Although the lottery randomized many thousands, the analyzable study ended with roughly 6,000 in the control group and 6,000 in the treatment group (those permitted to apply), of whom only about 25%—around 1,500 people—actually enrolled in Medicaid, which is the small group whose effects the latest analysis estimates.
“you've got 6000 people in a control group, 6000 people in, call it a 'treatment group'... of whom only about 25% actually enrolled in Medicaid... that's 25% of the treatment group, or about 1500 people. So that's what the latest study is looking at.”
States should expand Medicaid to provide some assistance rather than none to poor people.
- Frakt predicts the Oregon study, like the Reinhart-Rogoff work, will mostly be used by state legislators and partisans to justify positions they would have taken anyway—some citing the null physical health results, others the depression and financial benefits—rather than genuinely shaping policy.
“It's kind of like the Reinhart and Rogoff work, where you can say: Did this study influence policy or was it just used to justify positions that would have been taken anyway?... you'll have this study cited by state legislators and others debating whether they should expand or not.”
- Universal, non-means-tested health subsidies—like the tax exclusion for employer-sponsored insurance—are perverse because rich and poor alike receive benefits, but the rich receive a bigger benefit since the exclusion is worth more at higher marginal tax rates.
“It's not only that rich and poor people alike are getting benefits. Rich people are getting a bigger benefit... Because they have higher marginal tax rates.”
- The Oregon investigators publicly pre-specified all their analyses before running them, which means the study was not a fishing expedition that data-mined for significant results—a methodological strength that makes its findings more credible.
“they actually pre-specified all their analysis. So, this wasn't a fishing expedition. They put online: Here's what we are going to do... Exceptionally impressive.”
- The real Medicaid expansion decision facing states is not whether to give poor people cash instead of insurance—that option is not on the table—but simply whether poor people get some assistance or none, and Frakt comes down on the side of providing some assistance.
“the actual choice here is not some other thing like giving people cash--that's just not on the table. The choice is whether people get some assistance or none--poor people. And I come down on the give-them-some side rather than none.”
- Oregon's decision to allocate roughly 30,000 Medicaid expansion slots out of 90,000 applicants by random lottery inadvertently created a randomized controlled trial of Medicaid access, which Harvard and MIT researchers seized upon to study its effects.
“they offered enrollment to people on a lottery basis--they opened a lottery... And while this was going on, some investigators at Harvard and MIT heard about it, on National Public Radio (NPR)... they said: Oh, wow, this is a randomized trial.”
Medicaid produced substantial mental health benefits and financial protection from catastrophic medical costs.
- Medicaid produced significant mental health benefits: a post-lottery rise in depression diagnoses (likely an access effect), a roughly 30% reduction in screening-positive depression over time, and improved self-reported mental health—with prior work estimating you would need to double income to match the mental well-being gain.
“found that the screening-positive rate came down 30%... they got a statistically significant improvement in self-reported mental health”
- Medicaid produced large, statistically significant financial protection: the rate of catastrophic out-of-pocket expenditures (over 30% of income) fell sharply from a 5.5% baseline, the likelihood of medical debt dropped about 20%, and the share of people borrowing or unable to pay medical bills was cut in half—all entirely expected for health insurance.
“out of pocket medical expenditures that were above 30% of income--so, this is their definition of a 'catastrophic expenditure'... from a baseline rate of 5.5%... The likelihood of medical debt came down by 20%, and the proportion of people borrowing money or not paying off medical bills was cut in half.”
Evidence should come from bodies of work using diverse methods rather than any single underpowered study.
- Frakt states that if the study were scaled up dramatically—tens of thousands per group run over 10-30 years—and still found no effect, he would change his mind, illustrating a willingness to update beliefs that he claims to genuinely value even when evidence overturns his priors.
“I actually like it when what I thought was true is overturned by some evidence that I'm convinced of. I actually really like that. I would have been more pleased to have this study come out and say: Well, we had the power to detect minuscule improvements and we couldn't even find that.”
- Economists generally do not perform statistical power calculations because their sample size is fixed by the available data (e.g., a limited number of countries and years in macroeconomics), so they simply analyze what exists and report what is or isn't significant—unlike medical researchers who must demonstrate power for NIH or VA studies in advance.
“economists don't generally do power calculations. In fact, before a year or two ago, I had never done one... in economics if you can't change the n, the size of your sample, there's no point in a power calculation.”
- No rational person should base a decision on undersampled results: when a study lacks enough sample for a question, its reported estimate is not much better than noise even though it has some sample, and whether a study is underpowered is an objective, computable property of any study.
“no rational person, would want to base a decision on under-sampling... on some questions here, what is reported is not that much better than noise... this is something you can just compute... and it's objective.”
- No single study is definitive; confidence should come from a body of work using different methods, data, and ideologically diverse researchers pointing in a similar direction, while still weighting each study by its methodological strength—so a randomized controlled trial like the Oregon study carries more weight than observational work.
“one study is not definitive; you need to look at a body of work... if they all are kind of pointing in the same direction... it really increases you confidence... Having said that, I think you also have to weigh the methodological strengths of each study.”
Insurance helps targeted populations who are sick or poor more than it helps healthy people generally.
- Because Medicaid removes the price of care—it is literally someone else's money—enrollees more readily seek preventive services, producing large increases in Pap smears, mammography, and a roughly 20% rise in women reporting they got all needed care.
“if you don't have to pay for care and it's "someone else's money"... you are going to much more readily go in and get preventative screening. So there are big reductions--or big increases... in women getting Pap smears... and mammography. A 20% increase in women getting "all-needed care."”
- Public assistance for health care has its largest impact precisely for people who are sick or poor; for most healthy people insurance does little for health, so the evidence is consistent with the idea that health care helps a targeted subset rather than the population at large.
“for people who happen to be sick. Or poor. And/or poor... Public assistance for taking care of their health care does have an impact... that's where the action is.”
- For diabetes specifically, clinical evidence shows that taking blood-sugar medications strongly improves blood sugar control, so if insurance facilitates access to those medications and people adhere to them, a measurable effect on blood sugar should appear—making diabetes one outcome where Medicaid could plausibly move the needle.
“We know clinically, from clinical evidence, that taking certain medications if you have high blood sugar or diabetes really affects those things... And if health insurance facilitates greater access to those medications, and people follow through, then you ought to see a result.”
- Medicaid increased total cost of care from all sources by about $1,200 per person per year, reflecting enrollees' increased use of services like mammography and other tests they now had access to.
“if you don't have to pay for care and it's "someone else's money"... you are going to much more readily go in and get preventative screening. So there are big reductions--or big increases... in women getting Pap smears... and mammography. A 20% increase in women getting "all-needed care."”
Medicaid's design and cost-effectiveness varies dramatically across states and differs fundamentally from Medicare.
- The Affordable Care Act originally mandated every state to expand Medicaid to 138% of the poverty level, with the coercive threat that non-compliant states would lose all Medicaid funding; the Supreme Court ruled in 2012 that this coercion was impermissible and made the expansion optional, so each state now chooses whether and how to expand.
“every state was mandated to expand Medicaid up to 138% of the poverty level... the coercion here was that if states didn't make this extension, the Federal government could withhold all funding for all Medicaid... the Supreme Court came out with a ruling last summer that said... Let's make it optional.”
- Public spending per person differs sharply across programs: Medicare runs about $10,000 per person and the forgone tax on employer-sponsored insurance averages about $5,000 per insured worker, while Medicaid delivers a fairly standard benefit package to poor people at only about $3,000-$4,000 per person.
“what we are spending out of public funds on Medicare per person is about on the order of $10,000 per person. And what we are forgoing in tax collection on employer-sponsored insurance is about $5000 on average... The cost of that, for a variety of reasons, is extremely low--$3000, $4000 per person.”
- Medicaid is effectively two programs combined: a federally mandated core for poor people who also fall into categorical groups (old, blind, disabled, pregnant), and an optional state expansion covering people who are merely poor without meeting a category—so the common belief that all poor people are covered by Medicaid is false.
“you can think of it as really two programs put together. One is that it is a Federally mandated program for people who meet certain conditions in addition to being poor... and these are people who are old, blind, or disabled, pregnant”
- Medicare is primarily a program for retirees aged 65 and older (also covering some disabled people), while Medicaid is associated with poor people near or below the federal poverty level.
“Medicare is a program for retirees, people 65 and older... Medicaid... that thing would be that it's a program for poor people”
- Medicaid programs differ so much across the 50 states plus DC that it is nearly impossible to generalize about them, but this diversity creates a natural source of variation that researchers can study.
“If you've seen one Medicaid program, you've seen one Medicaid program. There are 50 plus D.C., and they are all different in some way. So it's very hard to generalize without basically lying a little bit. And an advantage of that diversity, however, is that it can be studied.”