Coase, the Rules of the Game, and the Costs of Perfection (with Daisy Christodoulou)
What this covers
Daisy Christodoulou and Russ Roberts examine what happens when technology is deployed to perfect decision-making in sports—and what the failures reveal about how rules actually work. The conversation centers on VAR in football, where the introduction of video review to eliminate wrong calls has instead generated confusion, eroded trust, and drained the game of spontaneity. But the real argument extends far beyond refereeing: Christodoulou draws on Ronald Coase and Michael Polanyi's work to show that the pursuit of perfect, technologically-mediated decisions misses how people actually value what matters. The framework applies equally to education, measurement systems, and any domain where rules must govern continuous reality.
The conversation moves through several distinct territories. One is the Consistency vs Common Sense trade-off: perfect consistency requires written rules that remove human discretion, yet Tacit Knowledge and context often matter more than rule-books can capture. Another is the problem of imposing categories on continuity—drawing a line between offside or handball when the phenomena are inherently gradual, making language the binding constraint, not measurement. A third is incentive effects: Coase Theorem insights show that where you draw the line reshapes behavior in ways nobody anticipated, like defenders tucking hands behind their backs or footballers diving constantly. Christodoulou argues that Comparative Judgment—having humans rank paired examples without wordy rubrics—captures judgment quality better than written rules do, and she cites evidence that this method produces paradoxically higher objectivity than detailed scoring guides. The conversation also surfaces a deeper point: introducing scrutiny sometimes increases confusion rather than clarity, and the things football fans lost when VAR arrived—flow, spontaneity, emotional resolution—were never counted in the accuracy ledger.
Christodoulou argues that the pursuit of perfect, technologically-aided decision-making (as with VAR in football) backfires because it ignores the trade-offs, the continuous nature of reality, the limits of language, and the incentive effects of where rules draw their lines.
- Right decisions are not the only thing we value; spontaneity, flow, and simplicity are traded away invisibly
- Many sporting judgments treat continuous phenomena as categorical, and the dividing line must rest on imprecise language rather than measurement
- Comparative judgment can capture tacit knowledge better than written rubrics by avoiding the distortions that wordy rules create
Applying intensive technology or scrutiny to decision-making often worsens rather than improves outcomes by losing consistency, clarity, and human values.
- When more scrutiny is applied to a decision, you don't always get more clarity; you can actually get more confusion, and transparency can erode trust by engaging too much with uncertainty, especially when two authorities (a TV expert and an on-field referee) publicly disagree.
“when you apply more scrutiny, you don't always get more clarity: you actually potentially get more confusion.”
- Introducing technology to get 'more right decisions' in football revealed that accuracy is not the only thing fans value; the lengthy reviews trade off against the fluidity, spontaneity, and emotional joy of the game, things nobody realized mattered so much until they were lost.
“what we realized when VAR came in is actually that is not the sole thing we want to optimize for. That is actually not the only thing we care about. We care about so many other things, some of which we hadn't realized we care about.”
- Football's troubles stem from grafting 21st-century technological scrutiny onto rules drafted in the 19th century in a pre-technological era, replacing an emergent bottom-up process of applying laws with a top-down imposition of very wordy rules.
“These rules were drafted in the 19th century, in a completely pre-technological era... what we are now attempting to do is to graft kind-of 21st century technology onto this system of laws that was not designed for that level of scrutiny.”
- Current VAR is a relatively backward use of technology because it merely uses technology to obtain a second human opinion -- like a 1980s bank manager faxing a loan application to another branch -- whereas the better future approach is algorithmic decision support, not human decision review.
“it's actually a relatively backward use of technology, because what it involves is essentially using technology to get another human second opinion.”
- Handball could be adjudicated by crowdsourcing comparative judgments of many video clips ('which is more of a handball?') from fans, players, and officials to build a distribution, then drawing a line on that distribution, and finally using AI pattern recognition to place a live incident above or below the line -- shifting from top-down decision review to bottom-up decision support.
“you don't say 'That is a handball, that isn't,' you just say, 'Which is more of a handball? Of these two clips, which is more of a handball?'”
Continuous phenomena cannot be fairly reduced to categorical rules without arbitrary lines that create distortion and unintended consequences.
- The mirror-image cognitive distortion to the tyranny of the discontinuous mind is the anarchy of the continuous mind: assuming that because everything is on a continuous scale there is no real difference between the ends, when in fact there is a real difference between a fish and a human and between safe and unsafe -- so the burden of authority is sometimes to draw and hold a line despite knowing it carries uncertainty.
“the opposite--a mirror image--cognitive distortion, is the anarchy of the continuous mind. Which is where you think, 'Well, because everything is continuous... then there is no difference between one end and the other end.'”
- Many decisions try to impose a categorical (on/off, onside/offside, goal/not-goal) system on phenomena that are actually continuous, drawing an arbitrary line on a continuum where measurement error means two cases either side of the line may actually be very similar or even reversed.
“in many of these situations we're trying to impose a categorical system--on/off, onside/offside, goal/not goal, touched by your hand/not touched by your hand--using something that is in fact continuous.”
- Defining writing quality in words creates distortions, as when rewarding 'fronted adverbials' leads children to write nonsense like 'Forgettably, he crept through the darkness' and earn higher marks than genuinely better writing that lacks the rewarded feature -- the same incentive distortion seen when defenders tuck hands behind their backs because brushing the ball gives a penalty.
“my favorite example is: 'Forgettably, he crept through the darkness.' And so, that's the point where you have this incentive. And that student will then get a better mark than a student who has not used a fronted adverbial but has written something really very good”
- Everyone expected offside to be perfect for technological review because it appears binary (in front of or behind a player), yet it became the most controversial VAR decision -- a classic conservative unintended consequence -- because offside is essentially impossible for a human to judge accurately as it requires looking at three places at once.
“everybody thought that offside was perfect for technological review... Because it is a binary decision about whether somebody is in front or behind a certain player.”
Comparative judgment aggregated at scale captures tacit knowledge better than explicit rules, definitions, or rubrics ever can.
- Human beings are poor at absolute judgments (e.g., 'how tall is that person?') but much better at comparative judgments (e.g., 'who is taller, left or right?'); aggregating many paired comparisons through an algorithm (Thurstone's law of comparative judgment) can build a reliable measurement scale.
“as human beings, we are not very good at making absolute judgments. We are much better at comparative judgments.”
- Language is not designed to deliver the precision we crave, which is why number was invented; tacit knowledge means we know more than we can tell, so fans know a handball when they see it and largely agree, yet defining it in words is extraordinarily hard -- as shown by the handball law growing from 11 words to eleven times that length without producing any more clarity.
“language is not designed to give us this precision. And, in a sense, the history of the invention of number... is almost a human invention to give us something that gives us the precision that language does not give us.”
- Paradoxically, having teachers make open-ended comparative judgments of student writing with no rubric produces very high agreement (objectivity), while marking the same essays with a detailed rubric and tick-lists produces low agreement (subjectivity), because comparative judgment captures tacit knowledge of quality that prose rubrics fail to specify.
“when you get people to make these judgments, in what seems this incredibly subjective way, with no click list, with no mark scheme, with no rubric... they have very, very high levels of agreement.”
- Comparative judgment is not mere rank-ordering; by linking large crowd-sourced assessments over time (e.g., 50,000 pieces of writing judged by thousands of teachers, each session linked to the last) it builds a robust distribution on which you can draw a line wherever your standards dictate, and it is especially good at holding that line consistently over time -- one of the hardest things to achieve in assessment.
“the number one misconception we have about comparative judgment, that people say it's all just rank ordering. It is so much more sophisticated than that.”
When you cannot quantify competing values, optimizing on one axis eliminates all others, leaving you with neither good nor bad outcomes.
- When you can't put a number on competing values you can't optimize on a curve, so you aim for the best of both worlds but risk ending up with the worst of both, like racehorses bred for speed until their bones get so light they break and they have zero speed -- you give up everything for nothing.
“the worst outcome is when you get the worst of both worlds. And, I talk about that in the evolutionary biology analogy as well.”
- Drawing on St. Augustine's concept of disordered loves, good things like the love of food, wine, or sport become idolatry when elevated into ultimate things and given the wrong priority; even non-religious people can accept that there should be an ordering and priority among the things we value.
“the love of food is a good thing. The love of wine is a good thing. If it becomes the ultimate thing, if it becomes the only thing, that's obviously very bad. You have things in the wrong order.”
- Many useful metrics invented by economists -- like quality-adjusted life years for valuing drugs or the micro-mort (a one-in-a-million chance of death) for comparing risks -- are better than nothing but lack the centuries-honed precision of physical measurements like height, mass, and temperature where measurement error has been reduced to irrelevance for everyday life.
“There's one that's used to often decide whether new drugs are worthwhile--so, it's quality-adjusted life years. There's another one--we talked about risks and safety--the micro-mort, which is the chance of something leading to a one in a million chance of death.”
- Footballers are paid vastly more than nurses or firefighters not because of any moral hierarchy but because we are collectively obsessed with football; we know the third-best young goalkeeper but not the best nurse in London, so the fans, not the players, are the source of the misallocation.
“the reason footballers get so much is we're all obsessed with it.”
- Equivalent review technologies work well and are broadly accepted in American football, cricket, rugby, and tennis (where Hawk-Eye works so well that line judges are being removed), making football's VAR a striking outlier that keeps failing despite repeated tweaks.
“you've got equivalent systems in cricket and rugby, which work very well. Tennis, you have the Hawk-Eye System, which has worked so well they're actually getting rid of line judges in a lot of tournaments. But, football just seems to be this outlier.”
- Before technology, the constant refrain in football was that everyone simply wanted more right decisions, since players' and managers' livelihoods depend on them in a billion-dollar industry.
“the thing you heard people say all the time, is that we just want more right decisions.”
Consistency and discretion are fundamentally opposed; granting individual judgment discretion produces inconsistency while rules sacrifice common sense.
- The core problem in traditional assessment is not disagreement between markers but disagreement within the same marker: in a study giving people 100 essays one week and 100 the next where six are secretly repeated, the same person gives the repeated essays different marks -- so the problem cannot be solved by hierarchy or an expert, because the expert will disagree with themselves.
“giving people 100 essays to mark one week, and then the next week, give them another 100 to mark. And, what they don't know is six of the essays are the same both times.”
- A fundamental trade-off in decision-making is consistency versus common sense: a common-sensical approach requires giving individual decision-makers discretion, but discretion produces inconsistency and accusations of bias, which is why sentencing guidelines exist for judges and why handball calls feel inconsistent.
“the trade-off at the heart of--a lot of decision-making is consistency versus common sense.”