Jennifer Doleac on Crime
What this covers
Jennifer Doleac of Texas A&M discusses her research on crime and criminal justice with Russ Roberts, examining how policies intended to reduce crime or improve policing often fail or backfire in practice. The conversation centers on causal effect measurement as the lens through which to evaluate interventions: rather than judging a policy by its aims, Doleac argues that rigorous empirical analysis must measure what actually happens, identify the full range of consequences, and always compare outcomes to the flawed status quo rather than to an imaginary ideal system. She draws on her own work and that of colleagues studying DNA database expansion, body-worn cameras, prisoner reentry programs, and sentencing practices to show how good intentions repeatedly produce surprises.
The asset covers a broad range of specific cases and mechanisms. On deterrence, Doleac presents evidence that increasing the perceived certainty of being caught deters crime far more than lengthening sentences—a finding from natural experiment work using database eligibility thresholds. On employment policy, she describes how Ban the Box legislation, designed to help ex-offenders, instead triggers statistical discrimination and reduces hiring of young Black men without college degrees. On surveillance and safety, she examines the tension between crime reduction benefits and privacy costs, the failure of body cameras to change officer behavior in Randomized Control Trial studies, and the Peltzman Effect—the risk that safety measures encourage riskier behavior. Throughout, she stresses that economists' distinctive contribution lies in their willingness to measure tradeoffs, calculate benefits against costs, and treat unintended consequences as reasons to refine policy, not to abandon measurement itself.
Doleac argues that criminal justice technologies and reform policies must be evaluated by their measured causal effects and full set of tradeoffs rather than their intentions, because well-meaning interventions frequently backfire and the relevant comparison is always the flawed status quo, not an ideal system.
- Increasing the probability of being caught (e.g., DNA databases) deters crime far more powerfully than longer sentences
- Ban the Box and body camera policies show that policies adopted for good reasons often fail to produce or even reverse their intended effects
- Economists' focus on causal identification and tradeoffs is what distinguishes their contribution from other disciplines
Policy impact must be rigorously measured through causal inference to weigh benefits against costs, not assumed from intentions.
- Policymakers should assume everything they try will fail—not from pessimism but to avoid becoming invested in pet programs before knowing they work—because the fastest way to find what has the biggest impact is to implement things in a testable way and abandon what isn't working, whereas agencies that believe in a program don't want to learn it fails.
“We should assume everything we try will fail. And it's not coming from a place of pessimism... we are slowing ourselves down by not... knowing that they are working... as soon as we figure out it's not working, move--try something else.”
- What distinguishes economists from other social science disciplines studying crime is a focus on identifying the causal effect of a policy and a toolkit developed for that purpose, because you must nail down a program's impact in order to calculate its benefits and weigh costs against benefits—a comfort with tradeoffs that other disciplines, especially public health, find hard to accept.
“we are much more focused than, I think, any other social science discipline on identifying the causal effect of a policy. You have to know you've nailed down what the impact of the program is, in order to calculate what the benefits of the program are.”
- Because empirical economics involves running many regressions and learning along the way (so the first regression isn't necessarily the best), the profession now expects authors to show in extensive appendices every alternative specification—dropping one state at a time, different thresholds—so reviewers can verify the researcher didn't choose the outlier, with reputation and trust as the ultimate safeguard against cherry-picking.
“what we've come to do as a profession is expect, for every judgment call you make, you better show me in the Appendix a table where you do it every other way. So I can see that you didn't just choose the outlier.”
- What economists can contribute to the risk-assessment debate is measuring the impact on ultimate social outcomes we care about (recidivism, racial disparities in sentencing) when the tool is implemented through the judge's actual decisions, rather than getting bogged down in whether the algorithm itself is accurate, because we never know the right answer in any individual case.
“what economists can contribute here is measuring what the impacts are on those ultimate outcomes rather than getting really bogged down in the details of whether the algorithm is accurate.”
Well-intentioned policies often backfire in unintended ways that empirical study can reveal and help address through better design.
- Ban the Box policies, which prohibit employers from asking about criminal records to help ex-offenders, backfire because they don't address why employers worry about hiring people with records; employers respond by guessing and statistically discriminating, reducing hiring of young black men without college degrees by about 5%—a clear case of unintended consequences from well-meaning policy.
“when they are not allowed to ask any more, they try to guess. And then they reduce their hiring of black men. And so you see a net reduction for young black men who don't have a college degree... their employment rates fall by 5% in the years after Ban the Box goes into effect.”
- Recognizing an unintended consequence does not mean a policy shouldn't be implemented; having a full understanding of all costs and benefits lets us implement other policies that mitigate those costs or find a better policy, whereas settling for the first thing that seems to work means missing opportunities to do even better.
“recognizing these unintended consequences doesn't necessarily mean we should... shouldn't implement the policy... having a full understanding of all of the costs and benefits in a situation--allows us to implement other policies that maybe can help mitigate those costs.”
- Body cameras could actually increase use of force because officers normally back off whenever there's any question in order to avoid making the front page, but when a camera will show the facts were on their side, they may feel freer to use force more often, which is one mechanism for policy backfire.
“most officers try very hard all the time not to make the front page of the newspaper. And, like, when in doubt, whenever they think there might be any question, they back off... when they are wearing a camera and it would be clear to anyone watching the video that the facts were on their side, then they might use force more often. And so that might be how you get this kind of policy backfire.”
- Holistic wraparound reentry services with intensive case management—the most praised and common intervention for ex-offenders, heavily funded by government—turn out in randomized controlled trials to have no benefits, and the treatment group was actually more likely to wind up with another conviction; possible mechanisms include the full-court press consuming time better spent job-hunting or eroding the person's sense of agency.
“when you actually do a Randomized Control Trial of them, they don't have any benefits. And, in fact... people who were in the treatment group who got access to the services in these programs that got all the money were more likely to wind up with another conviction down the road. Not less.”
- Randomized controlled trials of body-worn cameras, including the major Washington D.C. study, find essentially no effect on officers' behavior across a wide range of measures; across studies some find increases in use of force and some find decreases, averaging to no effect, demonstrating that policies implemented for good reasons don't necessarily have the impacts we expect.
“the end result was that nothing seemed to change. There was just like, no result, all across the board. This, it turns out, is in line with previous studies, more or less... Some find that body-worn cameras increase use of force, actually. Some find that it decreases use of force. On average, there's no effect.”
Technologies that increase certainty of apprehension deter crime more effectively than harsher sentences, though they impose privacy costs.
- Many crime-fighting technologies work by being core surveillance tools that increase the likelihood of getting caught, which inherently means keeping better tabs on everybody, generating privacy costs that are extremely difficult to quantify even though economists can at least calculate the benefits to inform the conversation.
“a lot of the tools are--really, they are core surveillance tools. And they could make policing and crime-fighting more efficient and productive by increasing the likelihood that you get caught for your crime. Which means at its core, they are keeping better tabs on everybody. And so that has privacy costs. And those privacy costs are extremely difficult to quantify.”
- Using natural experiments from database expansions in the US and Denmark that compare offenders just before and after a law makes them eligible for DNA collection, Doleac finds people added to the database are dramatically less likely to re-offend (about 40% drop in Denmark), revealing the power of increasing the probability of getting caught over increasing the sentence, partly driven by offenders overestimating how fast the technology catches them—a 'CSI effect.'
“what we find, in both the United States and in Denmark, is that people who are added to the database are dramatically less likely to re-offend. So, recidivism evolves[?] a lot. Sort of much bigger effects than I ever would have expected going into these projects.”
- One settled finding in the economics literature is that more police equals less crime: study after study with good causal identification shows that hiring more police officers reduces crime rates, and the best evidence suggests most American cities are under-policed rather than over-policed, though this holds only on average.
“more police equals less crime. We know from study after study after study that when you have good causal identification, hiring more police officers reduces crime rates... the best evidence suggests that most American cities are under-policed, not over-policed.”
- Technology is transforming the criminal justice system through recent advances like ubiquitous surveillance cameras, GPS monitors for people on probation, and DNA databases, which can make crime-fighting more efficient.
“having cameras everywhere... GPS monitors for people who are, yeah, on probation... DNA databases. Just a whole bunch of things... that are all relatively new advances, and that can make our crime-fighting more efficient.”
Imperfect tools that improve on the biased status quo move criminal justice in the right direction despite limitations and risks.
- A benefit of DNA database technology is that it spreads the risk of wrongful conviction to a broader group rather than concentrating it on the communities that are the usual suspects for police, because scientific matches make most arrests more accurate, and the relevant comparison is the status quo where police arrest the wrong person all the time, not an ideal system where the truth is known.
“one of the benefits of this technology is spreading the risk of wrongful conviction... to a broader group. I think it's healthy that we are all thinking now about, like, 'Well, what if I get caught up in the DNA database?'”
- Risk assessment algorithms in sentencing don't have to be perfect to be valuable; they just have to be better than the status quo, because if they remove some of the known biases in the current system (officers operating on hunches, a judge in a bad mood after his football team lost), they move us in the right direction even if not to an ideal endpoint, though they risk baking in existing biases with a veneer of scientific credibility.
“We don't have to come up with a tool that's perfect; we just have to come up with a tool that's better than the status quo.”
- The information actually stored in a criminal DNA database is not sensitive—it is essentially a string of numbers akin to a Social Security number, produced mechanically from a saliva swab—so the government is not using genetic material to determine health predispositions, even though people worry about a slippery slope toward such use.
“The information that's actually in the database is--there's no sensitive information. It's essentially a string of numbers. You can think of it as akin to a Social Security number. It doesn't really contain any information in and of itself.”
- Body cameras may be worth their cost for capturing the rare, horrifying event (an officer-involved death) and for community trust, but these benefits will never show up in a randomized controlled trial because such events are statistically too rare to register in day-to-day behavior, so policymakers must be clear about which goals they actually care about and whether their studies can measure them.
“that is a completely reasonable goal, and a completely reasonable thing to pay a lot of money for. But it will never be picked up in an RCT.”
Incarceration policy should scale back given evidence that marginal incarceration harms employment, though context and prison quality matter greatly.
- Evidence from Mike Mueller-Smith's natural experiment using random assignment of harsh versus lenient judges shows that for people on the margin of incarceration, being incarcerated harms their outcomes (employment decreases), suggesting the United States is incarcerating too many people and could scale it back, though juvenile evidence is mixed and effects depend heavily on prison programs and outside options.
“for people in that situation who are really on the margin of being incarcerated, where it really comes down to which judge you saw, incarceration harms their outcomes. So, employment decreases.”
- The effect of incarceration on the incarcerated person varies dramatically by context—Norwegian prison functions like a job-training camp that leaves people much better off, while US prison harms marginal adults—so the impact depends on what the programs are inside the prison and what the outside options and alternatives are for those not incarcerated.
“for people who were sent to prison in Norway, they are much better off when they get out. It's essentially going to, like, job training camp for a year or two, and you get all these incredible services, and you come out a much better person”
Economics disciplines researchers to test hypotheses against evidence rather than rely on priors or moral conviction alone.
- A strength of economics is that the profession rarely lets members coast on their priors: almost no hypothesis is off-limits in a seminar room, and disagreement requires showing evidence in data or a model rather than calling the questioner a bad person, which makes the research better and inoculates economists against moral persuasion.
“we rarely let each other coast on our priors... almost no hypothesis is off limits in an economics seminar room... you need to be able to show in the data... You can't just call them a bad person; and that's not going to end the argument.”
- The economist's response to the claim that any life-saving policy is automatically worth it is that the way we all live our lives—driving cars, doing risky things daily—reveals that we place a non-infinite value on lives, so weighing tradeoffs requires putting numbers on them, which is the nature of the game even though other disciplines resist it.
“the way we are all living our lives suggests that we don't place infinite value on other people's lives... we drive cars. We do things all day long that might potentially have negative impacts. And so obviously we are weighing tradeoffs, and we place some non-infinite value on these things.”
- Body cameras capture vast amounts of everyday interaction data that has enormous untapped potential to improve policing day-to-day, analogous to coaching teachers, but using it productively requires running footage through machine learning algorithms because it is economically infeasible for humans to watch all the footage—video storage costs fall while human labor costs to review it only rise.
“it is not possible in these departments for people that just, like, watch all of the video... the storage costs for the video will fall dramatically as the years progress. The human, like, labor costs of having someone sit there and watch all the hours of footage to see if anything went wrong--that is only going to get more expensive.”