What this covers

Mark Zuckerberg and Lex Fridman discuss the technological and social dimensions of Meta's push toward immersive computing. The core argument centers on Codec Avatars and mixed reality as tools for erasing the distinction between physical and digital experience. Rather than video transmission, Codec Avatars encode facial expressions directly, achieving photorealistic presence while reducing bandwidth demands. Mixed reality overlays digital objects—holograms, games, interfaces—into the physical world until the boundary between them dissolves. Zuckerberg frames this as a continuation of how humans share content, from text to photos to video, with immersive co-presence as the next natural frontier. He also outlines Meta's approach to AI: rather than building a single superintelligence, the company is developing distributed AI personas—fictional characters, business assistants, creator avatars—that users can interact with across platforms.

The conversation touches on both technical feasibility and social consequence. On the technical side, Zuckerberg explains the engineering challenge of mixed reality pass-through (multiple cameras must map physical space to eye perspective in real time) and the distinction between training and programming AI models—why making a reliable AI persona of a real creator is harder than inventing a fictional one. On the social side, he addresses identity questions raised by avatar capture (which version of yourself is "real"?), grief and memorialization when deceased loved ones can be replicated, and the Harm Principle applied to digital spaces, where physical violence in games causes no actual harm but anonymity creates new forms of toxicity. The conversation also flags the Developer Flywheel—that adoption requires millions of users before builders invest seriously—and previews AI Studio, Meta's planned platform for user-generated AI personas.

Sharpest takeaway

Zuckerberg argues that photorealistic Codec Avatars and mixed reality will dissolve the boundary between physical and digital, making 'the real world' the blend of both and delivering a genuine sense of social presence at a distance.

  • Codec Avatars transmit encoded facial expressions rather than video, achieving photorealistic presence more bandwidth-efficiently
  • Mixed reality (Quest 3) overlays digital objects on the physical world so holograms become as common as physical objects
  • Meta is building many distributed AI personas rather than a single super intelligence

The argument · threads7 threads · 29 claims
0.85

Mixed reality blends digital and physical worlds into single coherent experience enabling true presence.

6 pointscentrality 5/5
  • Many VR experiences don't need full immersion, and rendering them in mixed reality—so users can still see the physical world around them—makes activities like shooter games or fitness more comfortable and safer because people are less worried about hitting furniture, while still preserving the 3D immersion.

    “by being anchored to and being able to see the physical world around you, it just feels so much safer and more secure.”

  • Calling only the physical world 'the real world' is increasingly mistaken; the real world is the combination of the physical and digital worlds coming together, and this technology's role is to merge what were previously separate—digital accessed only through screens—into a single coherent experience.

    “I actually think the real world is the combination of the physical world and the digital worlds coming together. But until this technology, they were sort of separate.”

  • The distinguishing value of the Metaverse over a phone video call is that it lets people feel physically present together and participate in shared activities—playing games, holding meetings, mixed-reality hangouts—rather than merely seeing each other on a screen.

    “The thing that you can do in the Metaverse that is different from what you can do on a phone is doing stuff where you’re physically there together and participating in things together.”

  • A fundamental problem with computers, TVs, and phones is that they keep you sitting still, but people are not meant to be that way—much of what makes us human is moving around, so the goal is to natively blend the immersive digital world with physical movement rather than siloing digital life to a small screen.

    “It’s where the human experience is a physical one. So it’s not just about having the immersive expression of the digital world, it’s about being able to really natively bring that together.”

  • Even in its current headset-based form, this will be by far the most human and social computing platform that has ever existed, because it combines immersive digital capability with the physical, present nature of human beings.

    “I just think this is going to be by far the most human and social computing platform that has ever existed.”

  • The novel thing about this technology is conveying a sense of almost physical presence: the only thing not physically real is being in the same place, while the intimacy, trust, and nonverbal communication that previously required flying to be together are delivered by short-cutting the laws of physics to provide the social and psychological benefits of presence to anyone in the world.

    “we’re basically short cutting those laws of physics and delivering the social and psychological benefits of being able to be present and feel like you’re there with another person”

0.66

Photorealistic avatars and emotion expression are technically feasible but raise unresolved psychological and ethical questions.

5 pointscentrality 4/5
  • The realism of avatars raises the possibility of interacting with loved ones who have passed away; there may be a balance where reliving certain memories helps grieving, but it could also become unhealthy, requiring study, and Meta already has protocols for memorializing deceased users' accounts while protecting private content like messages.

    “if someone has lost a loved one and is grieving, there may be ways in which being able to interact or relive certain memories could be helpful. But then there’s also probably an extent to which it could become unhealthy.”

  • Avatar emotion expression should be tunable per person, since some people may prefer an avatar that emotes more strongly than their actual face does—for instance Zuckerberg, who has a relatively stiff physical expression, might want his avatar to better convey how he is actually feeling.

    “Some people might prefer a version of themselves that’s more emotive in their avatar than their actual faces.”

  • Codec Avatars work by scanning a person's face and body across many expressions to build a computer model, so that the headset can send an encoded version of what you should look like rather than a full video, making them both photorealistic and more bandwidth-efficient than transmitting video, especially 3D immersive video.

    “it can basically send an encoded version of what you’re supposed to look like over the wire. So, in addition to being photorealistic, it’s also actually much more bandwidth efficient than transmitting a full video”

  • Because a captured avatar reflects how you looked at the time of capture rather than your current state (hair length, shave, weight), and because people may scan a 'best version' of themselves, the technology raises unresolved psychological and identity questions about which version is really 'us'.

    “the avatar is realistic in the sense of, this is kind of what you looked like at the time of capture, but it’s not necessarily temporarily accurate to exactly what you look like in this moment.”

  • Most of human communication, even when people are speaking, is not the words themselves but the facial expression and nonverbal cues, which is why capturing photorealistic expressions matters for delivering presence.

    “There’s all these studies that most of communication, even when people are speaking, is not actually the words that they’re saying. It’s the expression and all that.”

0.66

AI companions and personalized AI personas will enrich social interaction and creator economies at scale.

5 pointscentrality 4/5
  • AI will make non-player characters in games much better, and AI companions like an adventure dungeon master—internally a favorite among Meta's testers—can deeply enrich the gaming experience.

    “I do think that AI will make the NPCs a lot better in games too.”

  • Creating brand-new AI personas (like a fictional dungeon master played by Snoop Dogg) is easier than building an AI that embodies a real creator, because the latter must reliably avoid saying things the creator doesn't want and represent things the way the creator would—AI models are trained, not programmed, so the challenge is making them predictable enough to stay within set parameters even if not perfect.

    “creating new personas is easier because it doesn’t need to stick exactly to what that physical person would want... If you want to actually make it so that you have an AI embodying a real creator, there’s a whole set of things that you need to do”

  • Creators could build an AI version of themselves—not after death but while alive—that interacts with their community to satisfy demand the creator cannot meet given limited hours in the day, helping fulfill both fans' desire to interact and the creator's desire to build community.

    “making it so that you could build an AI version of yourself that could interact with people not after you die, but while you’re here to help people fulfill this desire to interact with you and your desire to build a community.”

  • Llama 2 by itself is not a consumer product but a piece of infrastructure people build with; Meta's priority is fine-tuning it and its branches into consumer products—the AIs and personas—aiming for hundreds of millions and eventually billions of users.

    “Because Llama 2 by itself, it’s not a consumer product, right? It’s more of a piece of infrastructure that people could build things with.”

  • Meta is building a platform called AI Studio that over time will let anyone create an AI persona much like they create other user-generated content, with the full creative potential only emerging once the whole community can build these AIs across Instagram, Facebook, and other apps.

    “we’re also working on this platform that we call AI Studio that’s going to make it so that over time anyone will be able to create one of these AI almost like they create any other UGC content across the platform.”

0.56

Quest 3 reaches mainstream adoption threshold needed to trigger developer ecosystem and mixed reality flywheel.

3 pointscentrality 4/5
  • A mixed reality platform only triggers the developer flywheel—where the developer community begins experimenting and building in earnest—once millions of people are actually using it, which Quest 3 is expected to achieve in a way Quest Pro did not.

    “it’s only when you have millions of people using something that you start getting the whole developer community really starting to experiment and build stuff because now there are going to be people who actually use it.”

  • Unlike smartphones, where successive models are barely distinguishable, each new VR/MR headset generation delivers a major new capability—and the big shift now is mixed reality, the ability to superimpose digital representations of people or objects onto the physical world.

    “this isn’t like smartphones, where companies put out a new smartphone every year, and you can almost barely tell the difference... For this, each time we put out a new headset, it has a major new capability.”

  • Quest 3, priced at $500, will be the first mainstream mixed reality device, with mixed reality cameras and sensors multiple times better than Quest Pro and a stronger custom Qualcomm chipset than the Pro line despite costing a third as much.

    “it’s $500, and in some ways, I think the mixed reality is actually better in Quest Three than what we’re using right now in Quest Pro.”

0.53

Society must learn norms around digital harms, AI personalization, and freedom through experimentation and observation.

5 pointscentrality 3/5
  • Meta's vision is for the world to evolve so there isn't a single AI super intelligence, but rather many people empowered by a variety of AI tools to do their jobs and improve their lives, mirroring how we don't work with one person for everything.

    “we want the world to evolve in a way where there isn’t a single AI super intelligence, but where a lot of people are empowered by having AI tools to do their jobs and make their lives better.”

  • Society should be as free as possible, letting people do what they want unless they damage others or infringe rights, but the concept of harm differs digitally: actions that would hurt people physically (like shooting each other in a game) cause no real harm and should be freer, while new digital harms arise—anonymity weakens the repeat-game incentive to act like a good citizen, enabling toxic behavior.

    “the idea of damage is somewhat different in a digital environment.”

  • Business-oriented AIs would let customers talk naturally to resolve product issues, make reservations, or buy things, which benefits both businesses—who avoid staffing phone lines—and customers, who avoid waiting on hold or navigating websites.

    “having someone who you can just talk to in a natural way who can help you if you’re having an issue with a product or if you want to make a reservation... rather than navigate some website or have to call someone and wait on hold”

  • After a rigorous assessment and red teaming, Meta concluded that the value of open sourcing a foundation model like Llama 2 significantly outweighs the risks, and its reception—far more downloads and usage than expected—validated the decision.

    “I think at this point there’s the value of open sourcing, a foundation model like Llama 2. It’s significantly greater than the risks in my view.”

  • Society will collectively learn the norms around personalized AI—what's too far, what's fun, how much personalization versus generic is good—by putting these AI personas in people's hands and observing interesting failure cases at large scale, since nobody, including Meta, knows in advance what will work.

    “we have to all figure it out by using it, right?”

0.38

Media naturally progresses from text to photos to video to immersive present co-experience as communication frontier.

2 pointscentrality 3/5
  • There is a clear trend in how people express themselves and get content online—from text profiles in early Facebook, to photos, to today's predominantly video content—with immersive, present co-experience being the clear next frontier in this progression toward richer and more vivid media.

    “there’s a clear trend where over time the way that we want to express ourselves and kind of get insight and content about the world around us gets increasingly just richer and more vivid.”

  • Not too far off, perhaps by the end of this decade, we'll live in a world with lightweight glasses through which you'll see as many holograms when you walk into a room as there are physical objects, since any screen, media, book, art, or board game can be rendered just as effectively as a hologram.

    “maybe by the end of this decade, we’ll be living in a world where there are as many holograms when you walk into a room as there are physical objects.”

0.24

Mixed reality pass-through requires sophisticated multi-camera computational and AI mapping to user perspective.

2 pointscentrality 2/5
  • Mixed reality pass-through is not just cameras: it requires multiple cameras capturing different angles and a complex compute/AI problem to map that footage to your eye perspective in real time—because the cameras aren't where your eyes are—and then naturally superimpose digital objects.

    “it’s a pretty complex compute problem, an AI problem to map that to your perspective because the cameras aren’t exactly where your eyes are”

  • Meta's goal is to move from lengthy detailed scans to a quick two-to-three minute scan using just a cell phone—waving it in front of your face and making expressions—that produces avatars of the same quality as the full studio scans, which is one of the last remaining technical challenges.

    “the goal, and we have a project that’s working on this already, is just to do a very quick scan with your cell phone where you just take your phone, wave it in front of your face for a couple of minutes”