
Byrne Hobart - Optionality, Stagnation, and Secret Societies
What this covers
Byrne Hobart speaks with podcast host Dwarkesh Patel on the sources of technological and social stagnation, centered on a single diagnostic lens: coordination. Hobart argues that understanding how groups with hidden and divergent goals actually work together—or fail to—unlocks puzzles that seem intractable from other angles. The conversation moves fluidly between abstract financial concepts and concrete institutional examples, from the FDA's enforcement patterns to elite university admissions, from secret societies to the architecture of long-term wealth accumulation. Throughout, Hobart treats financial notions like optionality, expected value, and volatility as underused tools for mapping decisions in domains far beyond markets.
The episode covers ground in several directions at once. Hobart contends that productivity stagnation has two distinct components—the visible technological kind and a harder-to-measure decay in social technologies, from government effectiveness to the reliability of basic institutional responses. He explores why the pursuit of optionality has become systematically oversupplied, and how the rare decision to foreclose options might therefore signal value. On education, he argues elite schools remain individually rational to attend but collectively wasteful, and that the apparent conformity of their graduates reflects selection and incentive rather than institutional shaping. He examines how institutions persist despite failing at their stated missions as long as the problems they address keep growing. The conversation also touches on why rationalist communities have proven ineffectual at translating correct predictions into institutional change, whether conscientiousness will diminish in value as specialization rises, and what constitutes genuine secret societies versus their decay into credential mills. Throughout, the framing assumes that most human and institutional decisions can be understood through financial concepts most people overlook.
Hobart argues that the deep problem behind both technological and social stagnation is coordination — how groups with hidden, divergent goals collaborate — and that financial concepts like optionality, expected value, and selling-vs-buying optionality are underused lenses for understanding individual and institutional decisions.
- Most human and institutional decisions reduce to financial concepts (optionality, expected value, volatility) that others miss
- The pursuit of optionality is systematically over-supplied, so judiciously foregoing options is contrarian and valuable
- Productivity stagnation and institutional decline both stem from unsolved coordination problems
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
Last names prominent in 14th-15th century Florence remain more prominent today, with families more associated with being doctors then still more likely to be doctors now, showing socioeconomic persistence that outlives even direct loss of wealth.
“the last names that were more associated with being a doctor in um in 14th or 15th century florence those people are still more likely to be doctors”
A model must be used while remaining aware of its deficiencies: you put assumptions into a model to make predictions, and if you can't satisfy all those assumptions the model doesn't actually apply — so you should at least not be surprised to be surprised.
“you want to use the model but you also want to be aware of the deficiencies in the model”
Patient long-term compounding strategies (parking money to become the world's wealthiest in centuries) rarely work because the key risk isn't annual returns but the once-a-decade/century/millennium catastrophes—war, famine, revolution—that reset wealth to zero, especially for easily-seizable financial assets; the US is anomalous in never having had such a catastrophic drawdown.
“the thing you want to focus on is what are the once a decade once a century and if you're really thinking about it once a millennium risks that will turn up and that can actually push that return to zero”
Conscientiousness is mainly valuable for achieving goals set by someone else; as communication and transaction costs fall and more of the economy is mediated through software, there will be more specialized jobs, and people naturally interested in a niche won't need to be especially conscientious to succeed at it — though resisting increasingly addictive entertainment is becoming a more valuable form of self-discipline.
“conscientiousness is really important if you have goals that are set by somebody else and it's really what lets you achieve those goals”
Productivity stagnation has two components: technological stagnation visible in productivity stats, and harder-to-measure social technology stagnation (e.g. declining government effectiveness from the 1930s-60s to today); social technologies are invisible until they're gone, like discovering you can no longer trust that an emailed question gets a good answer.
“between technological stagnation which is pretty visible in the productivity stats and then social technology stagnation which is a lot harder to measure”
Elite higher education is probably still a good deal for the individuals pursuing it but a bad deal for society that so many pursue it; you should generally take the spot if you have it, but the one situation where dropping out is right is when there's a specific, concrete opportunity that requires it — and turning down easy prestige to succeed at something else is uniquely impressive both externally and internally because you've 'slain the optionality dragon.'
“i still view elite higher education as probably a pretty good deal for the individuals who are doing it i just think it's uh it's a bad deal for society that so many people are pursuing it”
Strict regulation in rich countries still effectively kills off whole technology avenues because only a handful of wealthy nations have both the spare wealth to indulge experimental hobbies and the developed capital markets needed to scale a proven product through its network effects; poorer countries where it may be legal lack the wealth and supply-chain capacity (e.g. only Boeing and Airbus make competitive wide-body planes, and China still can't match them).
“it does matter if regulations are strict in um in the handful of countries that are pretty rich and um and that also have developed capital markets”
Small-cap stocks are priced less efficiently than large caps partly because many financial institutions impose size cutoffs (e.g. ignore anything under $5 billion) since they can't take meaningful positions in small names, concentrating analytical talent on refining valuations of big companies rather than capturing obvious opportunities in small ones.
“small cap stocks tend to have less efficient pricing than large stocks and one of the reasons for that is that a lot of financial institutions have a size cut off”
The downsides of new technologies are easier to see than the upsides because risks are front-loaded to the early period when the technology is dangerous but doesn't work well yet (e.g. exploding early steam-engine boilers), whereas the large upside arrives decades later when it's mature and safe — so we don't get flying cars partly because we never see the costless future version, only the early accidents.
“that downside is easier to see because it is weighted to the very beginning when the new technology doesn't work especially well but it can kill people”
A portfolio of individually-selected stocks held over very long horizons will all eventually go to zero, but rebalancing over intermediate periods keeps the portfolio value positive — yet a national war or revolution can wipe out the entire value regardless.
“if you're just picking you know a set of stocks for example over a long time they'll all go to zero over more intermediate time periods you can rebalance and so that keeps the the portfolio value above zero”
Institutions can persist and their members stay important for a long time even when they fail to solve their stated problem, as long as the problem itself keeps growing — because members' unstated real goals are to keep their jobs, get raises, and feel important.
“if they fail to accomplish their goals but the problem they're working on keeps getting bigger they can stay important for a really long time”
Large institutions—public and private—reserve their fiercest enforcement for those who visibly and deliberately try to circumvent their rules; the FDA won't refuse a foreign miracle drug outright, but if it knows you specifically tried to ride around its regulations it takes that seriously, just as Disney pursues knockoff characters, because rule enforcement is both self-interested and a source of legitimacy.
“if they set up some kind of rule and you thwart it and that gets their attention then they're very serious about making sure that you don't toward it again”
The world contains little 'dark matter' requiring sinister coordinated forces to explain; far more often events are explained by incompetent forces with nobody in charge — you rarely see the chess pieces moving cleverly, you mostly see them moving at random — which itself implies the population of effective secret societies is low.
“there are a lot more times where you can look at the world and say wow there are pretty incompetent forces at work and nobody knows what they're doing and there's nobody in charge”
Society's growing preference for optionality (not starting a business, delaying family) is driven largely by a wealth effect — money buys freedom and choices, and subsistence societies have almost no optionality — combined with humans being wired to be more satisfied by pursuing specific goals than by keeping all options open, yet people pursue optionality fractally and get less done as a result.
“money is is a way to get freedom in in individual ways and then in aggregate ways and certainly you have a lot less optionality if you're living in a subsistence society”
For a 22-year-old, it is genuinely hard to do something with a higher expected/median outcome than getting a bachelor's from Harvard; more impressive paths exist but have lower expected value — so the credential is rationally chosen for its expected outcome, and only succeeding outside it (while turning it down) beats it.
“it is hard for them to do something that they expect to succeed in that is more impressive than getting a bachelor's degree from harvard”
Elite-school students' apparent risk-aversion is mostly a selection effect rather than schools making people conformist: going straight from a top school into consulting/banking/big tech is the conformist choice no one questions, and high-end admissions are now so randomized that those who got in feel they must stay on the exact safe path to remain 99.9th-percentile, while those just missing out feel achievement is hard and play safe.
“it's probably a lot of selection effect because i do encounter a lot of really bright people who did not go to elite schools and did really interesting things instead”
A major hidden function of an institution like MIT is to consistently supply motivation — peers and respected professors whose judgment makes you feel bad about C-minus work — which is why many people who attempt self-directed open-courseware equivalents overestimate their willpower; the residual market inefficiency is that conveying a credible skill signal to employers is hard, not that the knowledge is unavailable.
“one of the functions that a school like mit serves is just to consistently motivate you both by surrounding you with peers and by giving you professors who are very smart and whose judgment you respect”
We are at a genuinely strange moment because total factor productivity growth, after continuously accelerating (albeit slowly) from the dawn of time through the 1970s without prior unforced slowdowns, then decelerated — so either this is a unique slow-forever regime worth explaining, or we're in year ~10 of another 50-year deployment cycle whose lead technologies (like those of past industrial revolutions) will make their backers rich and give even non-rich participants interesting lives.
“we basically had growth that um continuously accelerated albeit at a slow pace from pretty much the dawn of time through the 1970s”
Closing the roughly 1%-per-year gap in total factor productivity growth, compounded over the ~50 years since the early 1970s, would have produced roughly 60-70% higher GDP per capita — but the gains would likely manifest as new products or longer health spans rather than simply 70% bigger houses or more cars.
“you compound that over a 50-year time period so you you probably get to something like 60 or 70 higher gdp per capita”
Rationalists correctly anticipated COVID without making big inferential leaps: accepting an R0 of ~2+ and an assumed ~2% case fatality rate, simple mental compounding plus the timing (Wuhan shutdown right after Chinese New Year, continued flights to the US and Europe) made it obvious the virus would spread everywhere and sicken many — the conclusion followed straightforwardly from taking the premises seriously.
“all you had to do was start compounding some numbers in your head and you quickly realized okay this is going to be everywhere”
In financial markets, systematically selling optionality is more profitable over time than buying it (despite drawdowns in crashes); analogously in life, because almost everyone seeks maximum optionality and almost nobody explicitly tries to cut off options, the rare decisions that require foregoing other options are likely to be undervalued and therefore good for the person who takes them.
“you make more money over time by systematically selling optionality rather than by systematically buying it and i suspect that something like that is probably true in the real world”
Studying history is easier than contemporary trends because outcomes are known, whereas contemporary writers carry agendas plus a meta-agenda selection effect: the people busy making something happen lack time to write, while those left behind do write, making contemporary data noisier — though contemporary topics have far more data; the right stance toward current events is to read them as a pseudo-historian asking what will be debated about this in 50-100 years.
“the people who are really busy making it happen don't have time to write books and the people who got left behind one way or another they do have time so there are a lot of selection effects that make the data noisier”
Knowledge has convex returns because retention—not just reading—is where the convexity lives: reading multiple books on one period from different angles (political history, biography, technology) makes ideas connect, and the most memorable moments come from catching something an author missed (e.g. Deng Xiaoping's biography mentioning container ships only once, despite containerization arguably driving China's industrialization), which creates a 'dialogue with the author' that cements the rest.
“it's not just about reading the book it's also about retaining it and that that's really where the convexity kicks in”
The big underlying question Hobart studies is how people coordinate to solve complicated problems, because the interesting problems can't be solved by a lone genius and institutions often have a stated mandate that differs from their real internal goals (e.g. SpaceX's Mars mission versus maximizing shareholder value), making coordination genuinely hard to study because double-talk is structurally necessary.
“how how do people coordinate when they're solving complicated problems because a lot of the interesting problems in the world you can't have one lone genius solve them”
Modern universities can be viewed as having evolved from something like secret societies — founders with peculiar ideas exerting maximal influence on people at impressionable ages and making them important so they shape society — and the secret societies within them (e.g. at Harvard/Yale) have decayed into boring credentialing rituals for investment-bank jobs.
“you're already doing a secret society conspiracy you're just calling it harvard or yale or something”
Secret societies are hard to sustain because they must both keep and reveal secrets: people recruited under one narrative who are later told 'that was fake, here's the real plan' may be true believers in the original story, so the society fails by recruiting people it then turns out were the wrong fit — the inverse of what happened with early al-Qaeda, where associates thought bin Laden was a loser to mooch off until he organized real attacks.
“you have to keep secrets but you also have to reveal secrets and um if if people join it because of one narrative and then you tell them once they're pretty high up okay that narrative is totally fake”
For deeply understanding a topic, books occupy a sweet spot in the production-cost spectrum: a long tail of subjects is worth the effort of a book but not a documentary, because books require mostly cheap typing on top of arbitrarily-deep reading (and readers spend more hours on a book than a movie), whereas documentaries require many people's production hours yet get compressed to an hour; tweets/livestreams are cheaper still but have low hit rates for deep understanding.
“there's a long tale of things where it is actually worth the time and effort to make a book and it's not worth the time and effort to make a documentary”
Because books are cheap and the internet distributes freely, a determined person can become close to one of the world's leading public-facing experts on a sufficiently narrow topic in a fairly short time by reading academic papers, chasing their citations, and consolidating it into a public body of work — which signals work ethic and ability while teaching the deeper lesson that the world is far more complicated than you thought, internalized only by repeatedly discovering you were wrong.
“it is really not hard if you were determined to be if you pick a narrow enough topic to be close to one of the world's leading experts on it in a fairly short timeframe”
Escaping the career/country middle-income trap means developing a hard-to-match skill (analogous to a country's branded products or monopolized indigenous technology, like the Japanese-monopolized steps in semiconductor supply chains) so you compete on something other than price; competing on price is fine when young and cheap to learn fast, but eventually you must differentiate because you can't stay perpetually cheaper than average.
“the escape is to have some set of skills that you are pretty confident nobody can match and that people know that you have and that allows you to compete on something other than price”
Hobart's optionality argument is not 'always minimize options' but a market analogy: since markets are pretty (not perfectly) efficient, you should expect a superior payoff from foregoing optionality only if you're doing genuinely good work in something you're actually good at — like a structured-products trader who picks which options to keep and which to sell rather than buying or selling optionality wholesale.
“if markets are not perfectly efficient you but they are pretty efficient you should expect that you can get a superior payoff but only if you are actually doing pretty good work”
Embryo selection for disease prevention is likely to be approvable (with the definition of 'genetic disease' broadening over time), but selection for enhancement traits like NBA-level height will be very hard to approve in the US; if it happens legally it will likely happen first in China, whose government is more open to genetic intervention.
“if that kind of thing does happen and it's legal it's probably going to happen in china”
Families are a more viable vehicle than recruited secret societies for multi-generational goals because parents can be honest with children over time and work up to revealing the real plan; multi-generational wealth-then-politics families (e.g. the Kennedys' conspiracy to get a Kennedy elected president) function as approximate secret societies.
“with families you actually if you you have some familial goal and it's going to take multiple generations so you can you can be more honest with your kids than with someone you're trying to recruit from the outside”
For a writer, the real writing happens during reading — winnowing ideas and forming connections — so that sitting down to type is more like taking dictation of a coherent narrative already in your head; most of a book's effort is reading and learning, with the final artifact a relatively small fraction.
“the writing process happens while i'm reading and then what i'm actually doing when i sit down and start typing is more like um it's uh i'm more you know just uh taking dictation”
Podcasts differ structurally from books: a book is like a lecture optimized for depth, while a dialogue creates breadth by connecting two people's overlapping-but-distinct idea clusters through tangents — so podcasts generate broader connections but books achieve more depth, and answering a hard question definitively (e.g. is inflation transitory) requires the book-writing mindset.
“a book is more analogous to a lecture... the podcast is a dialogue so there could be tangents and both sides are contributing things”
Rationalists are relatively good at predicting events but bad at influencing them because their cultural norm of blunt honesty attaches them to the factual consequences of predictions and detaches them from social consequences; taking ideas seriously means they extrapolate to natural endpoints and loudly affirm weird-but-true conclusions (simulation, cryonics), so any rationalist warning about COVID could be dismissed by association with their other 'absurd' beliefs — where the normal heuristic wrongly treats 'absurd' as 'false' and low-status.
“it's this attachment to the factual consequences of predictions and detachment from their social consequences that makes rationalists relatively good at predicting the future but comparatively worse at influencing it”
Finance and debugging are both rationality 'dojos' but of opposite temperaments: finance is the extroverted one (you must model other people's behavior and figure out why the counterparty is wrong, often due to institutional/balance-sheet constraints that explain inefficiencies like small-cap mispricing), while debugging is the introverted one (an exercise in humility and self-knowledge, since debugging is harder than writing the code, so writing the cleverest code leaves you unable to fix it).
“in finance part of what you're doing is you're trying to model other people's behavior you're trying to figure out not just um why am i right but why is the person on the other side of the trade wrong”
Deliberately picking up skills adjacent to your work that you're bad at is valuable even if you never master them: if you succeed you'll delegate those tasks, and knowing roughly how hard they are (e.g. learning to program badly while working with programmers) lets you ask better questions and recognize fictional timelines.
“if you have a job where you're not doing any programming but you're working with programmers you should probably learn to program even badly just so you can have some intelligent opinions and at least so that you know which timelines are totally fictional”
To learn a topic, the best approach is to buy a smattering of books taking different angles on the same subject — including fiction written in or about the period, because authors notice tangible details others miss — and read all of the above rather than seeking one optimal text.
“i would try to buy a smattering of different books that take different angles on the same topic and fiction that is written in the time period that you are interested in”
Longer health spans are probably a more solvable and more pressing problem than longer lifespans in terms of total human disutility.
“people would have longer lifespans or hopefully that we'd have longer health spans um which i think is a more probably more solvable definitely more pressing in terms of total human disutility”
The financial markets exist and you can buy optionality directly through stock options.
“in financial markets you can buy optionality very directly you buy stock options”