148 | Henry Farrell on Democracy as a Problem-Solving Mechanism
What this covers
Henry Farrell, a political theorist at Johns Hopkins, sits down with Sean Carroll to argue that democracy deserves credit not merely as a moral ideal but as a superior problem-solving apparatus. The conversation centers on how democracies, hierarchies, and markets each process information and navigate complex solution spaces. Farrell, drawing on work with statistician Cosma Shalizi, contends that democracy's bottom-up, diverse, and experimental character allows it to harness cognitive diversity and reasoned deliberation in ways that price signals and top-down command structures fundamentally cannot. The discussion ranges across how each institutional form—and the feedback loops that sustain or rupture them—shapes what problems get solved and which stay locked away.
The episode traces several distinct lines of argument. Farrell explains how James Scott's concept of high modernism and legibility reveals the information losses built into hierarchical organization, then contrasts this with Hayek's defense of markets as discovery mechanisms, while showing why Joseph Stiglitz's economics of information complicates both stories. He argues that reasoning as social justification—the human capacity to criticize others while remaining blind to our own flaws—becomes productive when structured into deliberative settings. The conversation also tackles why democracies remain vulnerable to information flooding, how machine learning reproduces old bureaucratic classification problems under new names, and why critics like Jason Brennan and Bryan Caplan who dismiss democratic judgment on grounds of citizen ignorance miss democracy's collective information-processing strengths. Throughout, Farrell emphasizes that institutional superiority is dynamic rather than absolute: the question is not which system ranks highest universally, but how each handles distinct shocks and how feedback loops in governance either reinforce or corrode democratic problem-solving capacity.
Farrell argues that democracy, understood as a bottom-up, diverse, and experimental information-processing system, is an underappreciated and dynamically superior mechanism for solving complex social problems compared to markets and hierarchies, because it harnesses cognitive diversity and the giving of reasons in ways that price signals and top-down command cannot.
- Democracy harnesses the diversity of human goals and perspectives to search a rugged solution space and avoid getting trapped at local optima that elites would lock in under autocracy.
- Markets (Hayekian price signals) and hierarchies each compress away information that human deliberation and reason-giving can recover.
- Real-world democracy depends on feedback loops that can either reinforce or, when disconnected, tear democracy apart.
Political and economic systems should be analyzed as dynamic information structures with emergent collective properties.
- Political science has far less discussion than it should of treating political processes as dynamic, complex systems; aside from a few figures like Charles Lindblom and parts of Herbert Simon's work, this kind of conversation has yet to get going.
“There isn't nearly as much discussion of this stuff in political science, as you might expect.”
- Following Herbert Simon, markets, democracies, and bureaucracies should be understood as structures of collective information rather than mere aggregates of fully-rational individuals; because human information-processing ability is extremely limited (unlike the unlimited rationality assumed in Econ 101 and game theory), people offload information-processing tasks to these collective structures, so even markets must be analyzed as institutions with ineluctably collective, emergent properties.
“he thinks about markets and democratic institutions and bureaucracies as being structures of information, in a sense, and of collective information”
- Institutions can be modeled as populations of beliefs, where everyone holds a slightly different belief about what the institutional rule is; variation in beliefs arises from differing heuristics, misunderstandings, deliberate subversion, or new ideas formed when different ways of thinking collide, and which variant beliefs spread to become a new institution can be modeled as a process of network contagion weighted by the relative power of elites versus non-elites.
“if we think about institutions as being these populations of beliefs, so everybody has a slightly different belief about what the institutional rule is in people's heads”
- Per James Scott's high modernism, hierarchical institutions of the 18th-19th centuries sought to make the world legible (e.g., Haussmann's redesign of Paris into grand boulevards) so populations could be organized, schooled, and policed; this legibility involves huge losses of tacit knowledge by re-segmenting the world to fit the state's own categories.
“A lot of what they're trying to do is to make the world legible... you start to build these grand boulevards... in order to make the people of Paris visible to the state”
- Machine learning ('high-tech modernism') has more in common with Scottian hierarchy than with markets, because both are fundamentally processes of classification and categorization; just as traditional bureaucracies sorted people with paper files, ML sorts datasets into classification schemes, and debates over ML bias are new versions of old fights like financial redlining.
“machine learning has a lot more in common with Scottian versions of hierarchy than it does with many of the... With the market notions that people very often apply”
- Following Hayek, the price mechanism serves as a process of discovery that acts as a summary statistic capturing signal from a vast set of latent production relationships that are fundamentally invisible or inarticulable to individual humans, allowing dispersed knowledge to coordinate without anyone being able to verbally explain it.
“the price mechanism becomes this extraordinary means of capturing signal out of a set of latent processes that are very, very... That are fundamentally invisible to human beings or at least, inarticulable by human beings”
Democracy harnesses diversity to discover better solutions than individuals or autocrats can reach alone.
- Democracy is not an optimization problem; the engineering temptation to think disagreement could be solved by finding an optimal solution is mistaken, partly because people figure out what their goals actually are in the process of pursuing them, making democracy a never-ending process of discovery and revision rather than maximization of a fixed objective.
“she said, and I think this is absolutely right, that democracy is not an optimisation problem”
- The ability of democracy to solve problems is underestimated because people overlook how democracy forces people with very different perspectives and wants to work together, and from the very diversity of goals, information emerges; democracy works best when it can harness that process and turn diversity into actionable solutions.
“the ability of democracy to solve problems is underestimated because people don't pay attention to the ways of which democracy forces people, ideally, with very different perspectives, very different wants to work together”
- Democratic problem-solving is best understood as searching across a rugged solution space where solutions are unknown in advance; the goal is to avoid getting trapped at a local optimum that is much lower than peaks achievable elsewhere, which is why modes of decision-making that can reach better optima matter.
“you're searching across a rugged solution space where you do not know what the solutions are... you want to make sure that you don't get trapped at some local peak, which is much lower than the peaks that you might be able to achieve otherwise”
- Under reasonable circumstances democracy has a dynamic advantage over autocratic and totalitarian systems in searching the solution space, because solutions beneficial to the collective majority get locked out in autocracies or highly unequal democracies when they are uncongenial to the powerful elite minority; correspondingly, more equal democracies are likely to be better at finding good solutions than less equal ones because elites are less able to block them.
“under reasonable circumstances, we can say that democracy is likely to have a dynamic advantage, vis-à-vis totalitarian systems, and its ability to search this landscape for possible solutions for problems that pop up”
- Following Mercier and Sperber, human reasoning evolved not as a way of figuring out the world but as a way of justifying our positions to each other in social contexts; this means we are good at seeing the holes in others' arguments but blind to our own, so well-constructed group settings with minimal willingness to self-correct can harness mutual criticism to converge on better solutions than individuals could reach alone.
“human reasoning evolves not as a way of figuring out the world, but as a way of justifying our positions to each other in a social context”
Rising inequality creates feedback loops that degrade democratic representation and obscure emerging social problems.
- As wealth inequality rises it feeds into the political process, creating feedback loops between wealth and political power so that many forms of democratic representation supposed to give people power no longer work; additionally, the divergence between comfortable college professors and people lacking economic opportunity creates an invisibility that hides long-simmering problems beneath the surface of American democracy.
“as you see wealth inequality going up, this begins to feed into the political process in some important way... you see these feedback loops developing between wealth and political power”
- There is a long-lasting independent tradition in US politics of using nominally knowledge-based voting tests as spurious justifications to stop people from voting; pre-Civil Rights era literacy tests were deliberately designed to exclude African-American voters, and such restrictions tend to come from the conservative side because conservatives historically feared an expanding franchise would be used to vote the rich out of their property.
“you very often had tests being administered to voters, which were very deliberately specifically intended to exclude African-American voters”
- A major problem for democratic decision-making is what happens when economic activities (such as complex financial products) become so complex that there is an incredible asymmetry of comprehension and even expert regulators cannot keep up with or understand what the market arrangements are actually doing.
“What happens when economic activities become sufficiently complex that it is impossible for regulators to keep up with them?”
Authoritarian rulers and digital platforms use information flooding and opacity to undermine democratic deliberation.
- Authoritarian rulers discovered that the best solution to 'more speech' is bad speech: by flooding the zone with bullshit they make it incredibly difficult for people to figure out what is happening, and combined with preventing the opposition from communicating and organizing, this gives rulers a substantial advantage; democracies, not just autocracies, are vulnerable to such information flooding.
“the best solution to more speech is bad speech, that is that if you flood the zone with bullshit, it becomes incredibly difficult for people to figure out what is happening anymore”
- Authoritarian regimes are notoriously terrible at taking feedback: they monitor citizens to stay in power, but when the feedback tells them things they don't want to know it is very hard for that information to penetrate the system; analogously, the open question for machine learning is what feedback loops could let people insert themselves to correct automated processes.
“authoritarian regimes are notoriously terrible at taking feedback... when the feedback tell some things that they don't really want to know, it's very hard to penetrate through the system”
- Machine learning differs from earlier bureaucratic legibility in being far more subtle: our pocket devices continuously feed data to be classified, and these classifications guide people into being the kind of person who gets sweet mortgage offers versus predatory loans, but this guidance is much more invisible than the direct forms of repression seen in previous eras.
“they guide you into being the kind of person who gets a sweet mortgage offers versus the kind of person who gets the kinds of predatory loans... but these processes of guiding tend to be much more invisible than the more direct forms of repression that we've seen in previous eras”
Hierarchy solves coordination problems but inhibits truthful deliberation and adaptive learning from diverse perspectives.
- Hierarchy's principal virtue is that it can get big complex problems solved: it provides a way to take a decision, communicate it to the actors implementing the solution, and ensure they implement it coherently without undercutting each other; its weakness is inflexibility and top-down loss of information, but its benefits include essential services like sewage and disease control that require central organization.
“hierarchies can get it done, is I think one way of thinking about it, that if you... want to get big, complex problems solved, at some stage or another, you probably want to have some degree of hierarchy in there”
- Hierarchy is poorly suited to the deliberative, reason-giving form of problem-solving because it is hard to offer compelling reasons to a boss who can fire you at a moment's notice, so much of the productive discussion gets short-circuited by the reality of power relations.
“it is really hard, for example, for you to offer good, compelling reasons for why your boss, who can fire you at a moment's notice, shouldn't have all the tomato sauce and hence, a lot of that discussion tends to get short-circuited by the reality of power relations”
- Athens, though a flawed slave society denying rights to women and metics, was far more dynamic than Sparta, able to adapt and import useful innovations; its more moderate power relations are not divorced from but connected to the flowering of figures like Socrates, Plato, Euripides and Aristophanes, demonstrating how a dynamic society with less extreme power disparities can be objectively culturally productive.
“it provides a way to see how it is that you can get a dynamic society where the power relations are much more moderate than they are in Sparta”
Understanding social complexity requires engagement with actual messiness, not imposition of alien formal models.
- Sparta is a clear example of strong autocracy: a small self-reproducing ruling class, a partially-rights-bearing Perioeci class, and a massive enslaved Helot population, with the whole society organized around keeping the Helots powerless; it survived for centuries but in an extremely static way, suspicious of innovation and refusing money, making it a particularly unpleasant society even for its elite.
“Spartan society is all organised in the same way... about keeping the Helot stand, keeping them from getting power... it managed to survive in an extremely static way, they don't allow the introduction of money, they are extremely suspicious of innovations”
- The population-of-beliefs model captures spontaneously arising informal institutions (like mask-wearing norms) much better than formal institutions (like whether the filibuster survives), and it shoehorns human creativity into a noise term in the equation, meaning the model deliberately omits much of what is genuinely ingenious about how humans manipulate their circumstances.
“it's much better capturing informal institutions... than it is at capturing formal institutions such as the question of whether or not the filibuster is or is not going to survive”
- Democracy is a dynamic system whose feedback loops can either reinforce or destroy it: politicians' selfish ambition reinforces democracy when channeled into appealing to voters but becomes dangerous when they refuse election results; citizen distrust of election systems is healthy in moderation but metastasizes into problems; and openness to new ideas brings benefits like Black Lives Matter but also harms like QAnon conspiracy communities.
“many of the things that have gone bad over the last number of years, our feedback loops which under other circumstances could be reinforcing for democracy”
- The early blogosphere created new social networks among academics who were among the few writing and thinking publicly about complexity-style ideas, which is how Farrell connected with collaborators like Cosma Shalizi and people at Santa Fe.
“you begin to have these different social networks. And I think it was through talking to people like you, talking to people like Cosma Shalizi”
- Because the strengths of political systems are dynamical, even if one system is better overall there may be specific shocks or problems it handles worse than others; we should think of institutions as flows and accept that hard universal rankings are impossible, settling instead for generalizations true across most reasonable circumstances that are more faithful to a dynamic, chaotic world.
“even if we think that one particular system is better overall, that there may be specific shocks, specific problems that it is going to encounter, which is going to do worse in dealing with than other systems”
- Physicists who parachute into social science with a workhorse model (like the Ising model) that superficially resembles a social phenomenon and then expect others to apply it will not generate real understanding; genuine insight requires engagement with social scientists already working in the area and with the actual messiness of real life.
“there's a long-standing history of physicists coming in and saying... 'Here is my workhorse model, I'm going to show you how my workhorse model generates something that kind of looks like what you're talking about'... That's not going to work.”
Market mechanisms coordinate dispersed knowledge through price signals but fail when complexity exceeds communicability.
- Following Stiglitz's economics of information, the price signal does wonderful things but simply cannot bear sufficient information to produce the nice efficiency results economists want from markets; Hayek's move of treating the most important knowledge as tacit and uncommunicable lets him defeat state planning but systematically precludes the possibility that people can solve problems through discussion in ways not reducible to price.
“you simply cannot expect the price signal to bear sufficient information to produce many of the nice results that economists would like it to produce in terms of having markets work well”
- Following John Stuart Mill, the chief dynamic benefit of markets is not the buying and selling itself but that world markets introduce people to a wide variety of different societies and approaches to problems; as markets approach anonymous spot-market relationships (like anonymous eBay transactions), they lose the ability to carry these coordinating and learning signals.
“the reason why markets are so beneficial to us isn't necessarily because of the buying and the selling, but because when you have world markets, people are suddenly introduced to a wide variety of different ways of knowing, of different societies, of different approaches to problems”
- Following Lindblom, markets serve a distinct function from Hayek's discovery story: they remove problems from the sphere of political argument, turning a contested political problem of distribution into one where people agree to disagree within a market allocation system, even though substantial inequality is baked into that arrangement.
“markets, in a sense, they remove problems from the sphere of political arguments”
Libertarian critiques of democracy and Hayekian opposition to state knowledge-gathering underestimate collective problem-solving.
- A set of influential libertarian critics (Jason Brennan, Bryan Caplan, Ilya Somin) argue democracy is doomed to fail for two reasons: individual citizens are ignorant and lack the political knowledge to make good decisions, and they are hopelessly cognitively biased, proposing alternatives such as epistocracy, greater reliance on markets, and the possibility of exit.
“democracy is doomed to failure for two reasons. First of all, because individual citizens are ignorant... And secondly, that they are hopelessly cognitively-biased”
- Per Quinn Slobodian's account, Hayek and his followers are committed both to a global WTO-style constitution acting as a straitjacket on states' ability to regulate property, and to strong opposition to the gathering of statistical and other forms of knowledge, which they see as a perversion that might give policymakers the false belief they can impose effective regulations.
“Hayek's disciples are very, very strongly opposed to the gathering of useful knowledge, statistical knowledge, and all of these other forms of knowledge, which they see as being a kind of perversion”
Institutional changes like voting reform and media accountability requirements can improve information quality and representation.
- Work associated with the Berkman Klein Center suggests that the worst disinformation in recent elections happened via Fox News, which acts as a conveyor belt hoovering up nonsense from the conservative fringes and injecting it into the mainstream of public debate; this implies media reform should target not only platform companies but also force traditional media to be more responsible.
“if you really want to see where the bad stuff happened in recent elections, it happened via Fox News, so that there is effectively a conveyor belt”
- Different voting approaches such as transferable (ranked-choice) voting could provide information benefits by making it easier for distinct parties on the right to emerge, helping to separate moderate conservatives from extremists; currently the bundling of crazies and moderates into one party forces people to subscribe to certain beliefs to remain in good standing, whereas a different voting system could let dissenters form their own party and isolate the extreme beliefs.
“there is a lot of argument to be made for different approaches to voting, transferable voting of one sort or another”
- One way to incentivize broadcasters toward truth without imposing strong gatekeeping is to oblige TV stations and broadcasters, as a licensing requirement, to maintain a significant fact-checking apparatus; this could be taken to imply a greater assumed responsibility, making it harder for them to escape liability for malicious slander, building on the legal liability Fox News faced over election-machine claims.
“by obliging as a licensing requirement, TV stations to have, and other broadcasters have a significant fact-checking, truth-checking apparatus as part of their make-up”