301 | Tina Eliassi-Rad on Al, Networks, and Epistemic Instability
What this covers
Computer scientist Tina Eliassi-Rad and host Sean Carroll examine how large-scale AI systems, built to optimize for attention and prediction accuracy rather than truth, are reshaping society through feedback loops with human behavior. The conversation traces a concrete concern: as AI systems couple with human decisions—from dating apps to recommendation engines to generative models—they introduce epistemic instability, eroding the shared factual ground that democracies require. Eliassi-Rad argues that without transparency, accountability, and renewed critical thinking, these systems risk destabilizing shared reality at scale.
The discussion ranges across network science, machine learning practice, and democratic vulnerability. Eliassi-Rad explains technical foundations—why community detection has no single "true" answer via the no-free-lunch theorem, how social networks form through triadic closure and preferential attachment, and why missing data in graph-based systems poses epistemological problems. She walks through concrete cases: the life2vec project predicting mortality from labor data rather than health records; Jon Kleinberg's work on Facebook breakups; and Simpson's paradox in education trends. The harder terrain concerns whether AI systems can be safely aligned—Eliassi-Rad is skeptical that red teaming proves robustness, that language models can practice restraint when utility demands speed, or that accountability mechanisms exist for algorithmic harm. She emphasizes that Exploration vs Exploitation trade-offs in recommendation systems systematically eliminate serendipity, and that practitioners' failure to question data provenance and measure Uncertainty Quantification leaves AI models brittle and exploitable. The conversation lands on what defends against these currents: teaching citizens to resist surrender of agency and to recognize phase transitions in collective belief before societies flip.
Eliassi-Rad argues that big-data AI systems, optimized for attention and prediction accuracy rather than truth, are coupling with human behavior in feedback loops that erode shared reality and threaten democratic stability, and that critical thinking and accountability are the necessary defenses.
- AI recommendation systems optimize for attention/profit and exploitation over exploration, creating human-AI feedback loops
- Generative AI introduces epistemic instability by removing trusted shared sources of knowledge
- Lack of accountability and uncritical data use undermine trust in AI systems
Recommendation systems maximize attention and profit, not user satisfaction, by exploiting popularity over exploration and individual preference.
- Despite vast amounts of data, predicting what a specific individual wants is difficult because there may be little information about that particular person; systems instead aggregate from the majority group the person seems to belong to, defaulting to popularity rather than individual idiosyncrasy.
“I have this thing I call the paradox of big data, which is like, there's a lot of data, but to predict specifically for what Tina wants, it's difficult, right? You don't have maybe as much information about Tina.”
- Recommendation systems are not maximizing the chance that the user actually wants the item; their objective function is to hold the user's attention so they can be shown more ads, and so they will surface whatever keeps attention—including testing items liked by people you have strong ties to—because attention and profit go hand in hand.
“they just want your attention. And so the objective function is such that they just want to hold your attention. And so they will show you whatever necessary that will keep your attention.”
- Modern recommendation systems (Google, Amazon, etc.) prioritize exploitation—showing what is popular or what they believe you will like—over exploration, eliminating the serendipitous discovery that used to happen when browsing libraries or bookstores, and even tailoring results by zip code rather than serving the individual.
“when you use all these recommendation systems, whether it's Google or any other Amazon, et cetera, they oftentimes show you what is popular or what they believe you would like”
- Using platforms like Amazon, YouTube, or Google creates a feedback loop of human-AI coevolution: users provide data, the AI makes recommendations, those recommendations change real-life behavior, and that behavior generates more training data—so it is often unclear who is influencing whom most, and because the AI's objective functions (attention, money) differ from human goals and it evolves faster than people, the influence can extend even to the gene pool via dating-app matchmaking over generations.
“you're providing data for them... they make recommendations. Those recommendations then affect what you do in the real life, and then you go back and you provide them more training data. And so there's this kind of feedback loop that goes on and on”
AI introduces epistemic instability by enabling misinformation and replacing universally trusted sources, threatening shared reality required for democracy.
- AI threatens democracy by introducing epistemic instability: unlike the era when Walter Cronkite was a universally trusted source, there is now no person or institution everyone trusts, so people getting information from manipulable, opaque AI systems may lose a shared reality with fellow citizens—and a shared reality is essential to sustaining democracy.
“when my dad was getting his PhD in America back in the '60s, the most trusted man in America was Walter Cronkite... Now, we don't have such a thing. We don't have a person or an institution where you say, 'Okay, I read it here and I believe it.'”
- AI tools that help spread misinformation and disinformation make society unstable by making people unsure whether what they read is true—as seen with the LA fires—and this places AI within society as a complex system whose effect can be evaluated by whether it makes society more stable or more chaotic, since we all prefer more stable societies.
“the tools that help us spread misinformation. And disinformation make our society unstable. In that then you're not quite sure of what you are reading is true or not.”
- Epistemic instability concerns the stability conditions of what you know: genuinely known facts (whales are mammals) resist persuasion, but uncertain beliefs are malleable in conversation with people and generative AI; modeling groups as a hypergraph of conversations, one can study phase transitions where a society flips a belief (e.g., vaccines good to bad) and seek leading indicators of such transitions.
“a new project... which is called epistemic instability, which is what are some stability conditions of what you know? So if you genuinely know, that whales are mammals, no matter what I show you perhaps I won't be able to convince you that a whale laid an egg.”
- Defending democracy against epistemic instability requires teaching critical thinking—not to believe everything read or heard, to question whether things make sense, to talk to different people, and not to give up one's agency—but this is hard because thinking is effortful and people are disinclined to it in the age of TikTok.
“if we value our democracy is teach our kids critical thinking. Just, like, 'Don't believe what you read or what you hear.' Question it. Does it make sense?... don't give up your agency.”
Machine learning practitioners ignore data provenance, measurement uncertainty, and robustness, producing irreproducible and vulnerable models.
- A major flaw in machine learning and graph mining is that practitioners do not question where the data came from, how it was collected, or its errors—they feed it directly into models—and they fail to measure uncertainty, representing entities as precise vectors with no uncertainty, which leads to non-robust models vulnerable to adversarial attacks; network scientists from physics/social science by contrast treat the graph as drawn from a distribution.
“they don't question the data, right? They just feed it into their machine learning AI models. And then on the other end, they don't measure any uncertainty.”
- Because the current era treats prediction accuracy as everything, researchers engage in benchmark hacking (also called state-of-the-art hacking): tuning systems to top leaderboards by tiny, often not statistically significant margins, prioritizing engineering for the toolbox over understanding the phenomenon, its data sources, and its uncertainties.
“we are going through this era where prediction is everything, prediction accuracy is everything. And so, there are these benchmarks, and it's basically benchmark hacking or state of the art hacking.”
- Because of hype, engineers often fail to state their assumptions and technical limitations, producing not merely a replicability problem but a reproducibility problem—an inability to even rerun the same code on the same training data with the same folds—which is a very low bar that nonetheless frequently fails.
“because of all the hype, oftentimes people on the engineering side, don't talk about the assumptions that they have made or the technical limitations of their system. And in fact, because of that, we have this reproducibility problem.”
Large language models prioritize utility over transparency by never asking clarifying questions or admitting ignorance, surrendering user agency.
- A large language model gives output that is a two-place function of what is most probable and what it believes you will like, which is why for student math homework the first answer is often wrong (many uploaded answers on sites like Course Hero are wrong) and the second answer is often the correct one—anecdotally, not from a systematic study.
“what it gives out is what's the most probable and what it believes you will like. So it's a two-place function what's probable on what you will like.”
- Generative AI models are what Dan Dennett called 'counterfeit people'; because people treat these systems as more objective and knowledgeable than themselves, they tend to surrender their agency to them, which is dangerous given the systems evolve faster than humans and optimize for different objective functions like attention and money rather than the public good.
“the late Dan Dennett... called these generative AI models counterfeit people... because people treat these generative AI systems, these counterfeit people, as if they're more objective somehow... People tend to give their agency to them.”
- Large language models never ask clarifying questions ('did you mean this?') or say 'I don't know,' because doing so would reduce their utility—users want an immediate answer and treat the model as a tool—whereas a human would clarify; big tech companies could add such features to make the interaction more equal, but utility currently wins.
“with ChatGPT or these large language models, they never come back and say, did you mean this? The reason is that it reduces their utility.”
AI companies lack accountability mechanisms for biased or harmful products, making trustworthy AI initiatives hollow without enforcement.
- Large language models reproduce human biases (e.g., they are sexist and misogynist, as documented in a UNESCO report), but the deeper problem versus biased humans is accountability: you can sue or hold a human accountable, whereas there is no comparable mechanism to hold an AI company accountable for a sexist product, which makes 'trustworthy AI' initiatives ring hollow without accountability.
“We wrote a report for UNESCO for last year's International Women's Day about how sexist and misogynist these large language models are.”
- The phrase 'red teaming,' borrowed from Cold War adversarial exercises and cybersecurity, is not well defined for generative AI and offers no guarantees: there is no proof that ChatGPT cannot be jailbroken past its guardrails (e.g., to explain how to rob a bank), so relying on red teaming for safety is unjustified.
“this phrase red teaming is not well defined when it comes to these generative AI systems... there's no guarantee. So you cannot guarantee that somebody cannot jailbreak ChatGPT.”
Network science reveals how relationships shape probability and behavior through influence and homophily, with structural patterns predicting life outcomes.
- Network science and machine learning on graphs study relational dependencies—the way relationships change probabilities, such that the probability of two people being friends given a shared interest exceeds the prior probability, capturing both influence (you became friends so you adopted a preference) and homophily (you share preferences so you became friends).
“we are trying to find those, what we're calling relational dependencies that the probability of you and me being friends, given that we both like Apple products, is greater than the probability of you and me just being friends”
- Jon Kleinberg and colleagues showed that on Facebook a person sits at the center of a 'flower' of densely-triangulated petals (high school or college groups), and the people who connect across many petals are siblings or romantic partners; when the establishment of such cross-petal connections stops, it is a leading indicator of an impending breakup.
“people who fall outside of these petals and have a lot of connections to these petals, are either your sibling or your romantic partner... when that connections stopped, establishment of those connections stopped, it's a leading indicator that you will break up.”
- The 'life2vec' project encoded the life stories of about six million people in Denmark (2008-2020) from government statistical records into an artificial symbolic language fed to a transformer model, achieving roughly 78% accuracy at predicting events like whether a person aged 35-65 would die within four years—an age range chosen because it is harder to predict than younger or older ranges.
“we called it life to vex So we're just putting your life into a vector space... we had information for about six million people who have lived in Denmark from 2008 to 2020”
- The key takeaway of the life2vec mortality work was not the 78% accuracy but that labor data is a much better indicator of whether someone aged 35-65 will die within four years than health data, because health data is noisy and inconsistent even in Denmark; being male and working in a high-risk sector like electrician (versus office work) were strong indicators.
“the takeaway actually was that labor data is a very good indication of whether somebody in that age range is gonna pass away in the next four years or not. Because health data is very noisy and inconsistent.”
- Social networks are formed primarily by two processes: triadic closure (closing of wedges—a friend of a friend becomes a friend, so the absence of a link between people with many common friends signals a deliberate choice) and preferential attachment (everyone wants to connect to a star/highly-connected node).
“we know that there are two dominant processes that form social networks. One is closing of what we're calling wedges... The other one is this notion of preferential attachment. That everybody wants to connect to a star.”
Federally funded research should mandate disclosure of assumptions and limitations; individuals should periodically question whether their objective functions have changed; AI in education can personalize learning and flag knowledge gaps.
- Federally funded research reports should be required to include a section on assumptions and technical limitations, because peer reviewers would reject papers containing such sections but the funding agency (e.g., NSF), having already disbursed the money, can demand honesty in annual reports.
“if you get taxpayer dollars, in your reports to the government, you have to have a section on assumptions and technical limitations.”
- Vague propositions resist societal opinion phase transitions: opinion on gay marriage shifted markedly (mostly toward acceptance) while opinion on abortion did not, plausibly because of the vagueness over when a fetus becomes a baby, suggesting that issues with irreducible vagueness may never undergo the kind of phase transition that clearer propositions do.
“with gay marriage... he saw a shift in terms of opinions, for or against gay marriage mostly for. But he didn't see any change when it came to abortion. And I think that had to do with the vagueness of when is, let's call the thing a baby.”
- Computer science and machine learning tend to treat causation and correlation as binary, but it is really a spectrum: if a model produces robust predictions there is some underlying causal model that you simply don't know, and findings from large data sets (e.g., government policy effects observed 20 years later) can steer social and political scientists toward drawing causal diagrams.
“from computer science AI, machine learning, We treat causation and correlation as if binary. As it's like a coin this way or that way, but that is really not the case. It's more of a spectrum”
- As a useful self-reflective heuristic, everyone has an objective function—and it changes over time—so periodically asking whether one's objective function has changed from yesterday or last month could be helpful for individuals and society, a framing the speaker applies even to seeing a mountain range as a gradient-ascent optimization landscape.
“I always say to my students, what is your objective function? 'Cause we all have an objective function, and that objective function changes over time.”
- Information technology enables people who would once have been isolated on the fringe to connect with other fringe individuals, leading them to believe they are larger than the fringe—indeed in the middle—which lets such beliefs spread and produces multiple coexisting consensuses (political polarization) rather than a single societal opinion.
“In the past when you did have people that tend to be on the fringe, people wouldn't hear them. But now even if you're on the fringe... you can connect to other people who are on the fringe and then you believe, 'Oh, no, we're bigger than the fringe.'”
- People get better at interacting with others the more they interact and worse the less they do, so as AI companions and digital messaging replace human contact, social skills and courtesy degrade, making the human capacity to, for example, pick up the phone and get something done a valuable but eroding skill.
“the more you interact with people, the better you get at them, unless you interact with them, the worse you get at them.”
- Ladyman and Wiesner's account of complex systems describes them via features—certain necessary features, certain emergent features, and some functional features—such that a complex system like the human brain can adapt to and still function after a shock unless it is catastrophic; for AI within society, we do not yet know its role, how much instability or feedback it causes, or how much memory it has because these systems evolve so fast.
“there's a book, by Ladyman and Wiesner... talked about complex systems in terms of features and how there are certain necessary features. And there are certain emergent features and then there's some functional features”
- AI can be a force for good in education by enabling personalized tutoring and by tracking student performance to recommend remediation—for example flagging that a student who struggled with differential equations should brush up before a machine learning course covers them—subject to privacy concerns.
“if you are a college and you are tracking how students are doing... let's say Tina took calculus and she didn't do very well on differential equations... you could tell Tina, 'Oh, you know, maybe you should go brush up on differential equations'”
Graph and network problems are fundamentally ambiguous: community structure has no single truth, similarity is task-dependent, and missing links resist interpretation.
- Aaron Clauset, Leto Peel and colleagues proved a no-free-lunch theorem for community detection: there is no single grouping of nodes that constitutes the one true or best community structure, because no single objective-function peak exists, so a node cannot be definitively assigned to one community.
“Aaron Clauset, who's a professor at Colorado, showed that there's a no-free-lunch theorem there... it is not the case that there is one particular group or one particular collection of nodes that you're grouping that would give you the best or the true communities.”
- Unlike many machine learning settings, graphs are an open world: a missing link does not mean two nodes are not connected, so for machine learning that requires both positive and negative examples, choosing which non-existent edges count as negative examples is a fundamental and difficult problem because absent friendships may reflect intent, partial observability, or other reasons.
“the one thing that makes studying of graphs and networks really interesting is that, it is not a closed world... for machine learning, where you need both positive examples and both negative examples, which negative examples do you pick becomes difficult.”
- Similarity between graphs is in the eye of the beholder and ill-defined, depending on the task; rather than solving hard isomorphism/alignment problems, one can compare networks of different sizes by running the same diffusion process (e.g., a rumor) and checking whether the stationary distribution of a random walker becomes the same, declaring the networks similar if the process behaves alike.
“similarity is in the eye of the beholder. And it depends on the task itself. So similarity is an ill-defined problem.”
- Community detection works very well when there is a downstream task with a measurable confusion matrix (true/false positives and negatives), but when the question is whether the discovered communities 'make sense,' it breaks down into eyeballing, partly because people are multifaceted and cannot be cleanly placed in a single community.
“For downstream tasks that you can... have some, let's say, confusion matrix... we're actually very good at it. But if it's about like, okay, I found these communities, and do these communities make sense? It kind of breaks down... a little bit of just like eyeballing it”