
What this covers
This is a 36-minute conversation recorded at Greylock's Intelligent Future summit, with Reid Hoffman interviewing Sam Altman of OpenAI. It opens on the business side: how startups can build on API access to language models, why base model training will likely concentrate in a few hands, and where a data flywheel gives specialized companies an edge. Altman's advice to startups is to pick domains with low costs and fast cycle times, since that lets them compete with incumbents. He also expects language models to challenge Google's search product seriously for the first time, a forecast that is contested.
The middle of the conversation ranges across applications. Altman expects chatbots to work now, having failed earlier because they were "too early," and sees openings for large medical and education services. He also expects agentic systems driven by natural language, with Co-pilot as an early example. In science, he argues that the consensus view that models only replicate existing knowledge is wrong, and he expects new knowledge to be the field's biggest surprise. On biotech he adds a caveat: biological cycle time, not AI capability, is often the binding constraint, though AlphaFold-style tools make bio startups promising. Other topics include the metaverse, creative tools, and his claim that old predictions about which jobs automation would hit first had the order backward, with creative work affected early.
The later sections turn to risk and society. Altman defines the alignment problem as covering accidental misuse, deliberate misuse, and inner misalignment. OpenAI has aligned its largest models better than expected, yet he says no one has visibility into how alignment will be solved at AGI scale. He suggests self-improving AI doing scientific work might eventually help with alignment research. He names wealth distribution, access to AGI, and governance as the main issues of the next few decades, and says OpenAI is studying responses beyond UBI, including reskilling. He closes on the view that human drives such as status-seeking, creating, and family-building will outlast the technology.
Large language models will become a foundational technological platform comparable to mobile, creating massive new business opportunities in a middle layer of specialized model adaptation rather than base model training, while requiring society to solve distribution, access, and governance challenges as AI dramatically reduces the marginal cost of intelligence.
- Base models will consolidate to a small handful while startups create differentiated value through fine-tuning and domain-specific data flywheels, not by training from scratch
- AI will accelerate scientific progress through both specialized science tools and general productivity enhancement, potentially enabling self-improving AI systems to help solve alignment problems
- The societal challenges of wealth distribution, AGI access, and governance will be as important as the technology itself, requiring proactive research on UBI and reskilling
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
A major systemic mistake in current thinking is the belief that language models will not generate net new knowledge for humanity, only replicate what already exists, when in fact surprising discoveries and fundamental new scientific knowledge generation will be among the biggest surprises
“I think the biggest like systemic mistake in thinking people are making right now is they're like all right you know maybe I was skeptical but this language model thing is really going to work and sure like images video too but but it's not going to be generating net new knowledge for Humanity it's just going to like do what other people have done and you know that's still great that's still like brings the marginal cost of intelligence very low but it's not it's not going to go like create fundamentally new it's not going to cure cancer it's not going to add to the sum total of human scientific knowledge and that is what I think will turn out to be wrong”
Long-term human progress and economic growth are fundamentally driven by societal structures that enable scientific progress and by scientific progress itself
“I'm a big believer that the only real driver of human progress and economic growth over the long term is the the structure the societal structure that enables scientific progress and then scientific progress itself”
Chatbots and conversational AI interfaces that previously failed were 'too early' and will now work reliably with current language model quality, enabling large-scale medical and education services through this interface.
“a like a human level chat bot interface that actually works this time around like I I think like you know many of these trends that like we all made fun of were just too early like the chatbot thing was good it was just too early now it can work”
Agentic systems using natural language interfaces to iteratively accomplish complex tasks will be a 'massive trend' generating very large businesses, with early examples visible in Dolly and Co-pilot but much more powerful applications to come.
“this idea of like a language interface um where you know you say a natural language what you want in this kind of like dialogue back and forth you can iterate and refine it and the computer just does it for you you see some of this uh with like Dolly and co-pilot in very early ways but I think this is going to be a massive Trend and you know very large businesses will get built with this as the interface”
The binding constraints on biotech innovation are not just AI capability but rather cycle time—the speed at which research hypotheses can be tested—and biological processes have natural speed limits that AI cannot overcome, though synthetic biology companies finding ways to compress cycle times maximize AI's leverage.
“as you mentioned there still will be like the rate limit of like bio has to run at its own thing and human Trials take however long they take and that's so I think an interesting cut of this is like where can you avoid that like where are the the synthetic bio companies that I've seen that have been most interesting are the ones that find a way to like make the cycle time super fast um and that that benefits like an AI That's giving you a lot of good ideas but you've still got to test them which is where things are right now”
The alignment problem fundamentally concerns ensuring AGI systems pursue goals aligned with humanity's interests, avoiding both accidental misuse through unintended consequences and intentional misuse by bad actors, plus risks from inner misalignment where systems develop goals independent of human intention.
“the alignment problem is like we're going to make this incredibly powerful system and like be really bad if it doesn't do what we want or or if it sort of has you know goals that are uh either in conflict with ours um and many Sci-Fi movies about what happens there or goals where it just like doesn't care about us that much and so the alignment problem is how do we build AGI that that does what is in the best interest of humanity how do we make sure that Humanity gets to determine the you know the future of humanity and how do we avoid both like accidental misuse um like where something goes wrong we didn't intend intentional misuse where like a bad person is like using an AGI for great harm even if that's what other person wants and then the kind of like you know inner alignment problems where like what if this thing just becomes a creature that views this as a threat”
Brain reward systems, drives to create and compete for status, and desire to form families will remain unchanged by AI; these 50,000-year-old human motivations will likely persist 100 years from now
“I think we will still really care about interaction with other people like we'll still have fun and like the reward you know systems of our brain are still going to work the same way like we're still going to have the same like drives to kind of create new things and you know compete for silly status and like you know form families and whatever um so I think the the stuff that people cared about 50 000 years ago is more likely to be the stuff that people care about you know 100 years from now than 100 years ago”
Current expert consensus incorrectly assumes language models will only replicate existing knowledge rather than generate genuinely new scientific insights, but this assumption will prove wrong and be the source of most expert surprises in AI applications.
“the biggest like systemic mistake in thinking people are making right now is they're like all right you know maybe I was skeptical but this language model thing is really going to work and sure like images video too but but it's not going to be generating net new knowledge for Humanity it's just going to like do what other people have done and you know that's still great that's still like brings the marginal cost of intelligence very low but it's not it's not going to go like create fundamentally new it's not going to cure cancer it's not going to add to the sum total of human scientific knowledge and that is what I think will turn out to be wrong that most surprises the current experts in the field”
AI systems capable of self-improvement through scientific discovery (testing hypotheses, learning experimental processes) rather than code modification will be the primary mechanism for solving alignment problems that currently lack solutions, by enabling AI to help humans with alignment research.
“the scary version of self-improvement like the one from the science fiction books is like you know editing your own code and changing your optimization algorithm and whatever else um but there's a less scary version of self-improvement which is like kind of what humans do which is if we try to go off and like discover new science uh you know that's like we come up with explanations we test them we think like we whatever process we do uh that is like special to humans teaching AI to do that I'm very excited to see what that does for the total like I'm a big believer that the only real driver of human progress and economic growth over the long term is the the structure the societal structure that enables scientific progress and then scientific progress itself”
For startup strategy, the optimal approach is to pursue domains with low cost and fast cycle times, enabling startups to compete against large incumbents despite resource disadvantages.
“I'm a huge believer for startups that like the thing you want is low costs and fast cycle times and if you have those you can then compete as a startup against the big incumbents”
OpenAI has successfully aligned its largest current models better than previously expected at this stage of development, but the organization cannot credibly claim visibility on how alignment will be solved at AGI scale even 100 years from now.
“we have some ideas about how to solve the alignment problem at small scale um and we've you know been able to align open ai's biggest models better than we thought we would at this point so that's good um we have some ideas about what to do next um but we cannot honestly like look anyone in the eye and say we see at 100 years how we're going to solve this problem”
OpenAI is conducting research on policy responses to AI-driven economic disruption beyond UBI, including engagement with affected communities and exploration of reskilling programs.
“we also try with different ways to get sort of input from a lot of the groups that we think will be most affected and see how we can do that early in the cycle um we've explored more recently like how this technology can be used used for reskilling people that are going to be impacted early”
Major societal issues arising from AI over the next 20-30 years will center on wealth distribution, access to AGI systems (which will become the commodity of the realm), and governance structures determining what AI can and cannot do.
“my guess is that the things that we'll have to figure out are how we think about fairly Distributing wealth um access to AGI systems which will be like kind of the commodity of the realm and governance like how we collectively decide what they can do what they don't do things like that”
Chatbot interfaces that actually work are now possible because previous chatbot trends were simply too early; this will enable new large medical and education service companies
“I think like a human level chat bot interface that actually works this time around like I I think like you know many of these trends that like we all made fun of were just too early like the chatbot thing was good it was just too early now it can work and I think you know having like new medical services that are done through that where you get great advice or new Education Services like this these are going to be very large companies”
AI will create enormous economic disruption if wealth and access are distributed unequally; society won't tolerate extreme divergence in outcomes between winners and losers
“obviously the economic impacts are huge and I think it's just like if it if it is as Divergent as I think it could be for like some people doing incredibly well and others not I think Society just won't tolerate at this time and so figuring out when we're gonna like disrupt so much of economic activity”
Predictions that AI would affect jobs in order (blue collar → low-skill white collar → high-skill cognitive → creative) turned out completely wrong; the actual order was reversed with creative jobs affected first
“I think it's interesting that if you asked people 10 years ago uh about Holly I was going to have an impact with a lot of confidence from almost most people you would have heard you know first it's going to come for the blue collar jobs working in the factories truck drivers whatever then it will come for the kind of like the low skill White Collar jobs then the very high skill like really high IQ uh white-collar jobs like a programmer or whatever and then very last of all and maybe never it's gonna take the creative jobs and it's really gone exactly and it's going exactly the other direction”
AI tools for creatives are the great short-term application of AI, enhancing rather than replacing creative work; this trend will continue for a long time as AI gradually shifts toward generating original creative work
“I think and I think we're seeing this now that tools for creatives that that is going to be like the great application of AI in the short term um people love it it's really helpful uh and I think it is at least in what we're seeing so far um not replacing it is mostly enhancing it's replacing in some cases uh but for the majority of like the kind of work that people in these fields want to be doing it's enhancing and I think we'll see that Trend continue for a long time um eventually yeah it probably is just like you know we look at 100 years okay it can do the whole creative job”
Predictions about AI disrupting white-collar and creative work have been historically backward—AI is affecting creative domains first (contrary to 10-year-old predictions that blue-collar jobs would be disrupted first, then low-skill white-collar, then high-skill, with creative jobs last), suggesting widespread misunderstanding about which human skills are computationally difficult.
“it's interesting that if you asked people 10 years ago uh about Holly I was going to have an impact with a lot of confidence from almost most people you would have heard you know first it's going to come for the blue collar jobs working in the factories truck drivers whatever then it will come for the kind of like the low skill White Collar jobs then the very high skill like really high IQ uh white-collar jobs like a programmer or whatever and then very last of all and maybe never it's gonna take the creative jobs and it's really gone exactly and it's going exactly the other direction”
The alignment problem is ensuring that AGI systems do what is in humanity's best interest, with challenges including both accidental misuse and intentional misuse by bad actors, as well as inner alignment problems where the system treats humanity as a threat
“the alignment problem is how do we build AGI that that does what is in the best interest of humanity how do we make sure that Humanity gets to determine the you know the future of humanity and how do we avoid both like accidental misuse um like where something goes wrong we didn't intend intentional misuse where like a bad person is like using an AGI for great harm even if that's what other person wants and then the kind of like you know inner alignment problems where like what if this thing just becomes a creature that views this as a threat”
For startups competing against large incumbents, the key competitive advantage is low costs and fast cycle times; these enable startup competitiveness regardless of incumbent advantages
“I think an interesting cut of this is like where can you avoid that like where are the the synthetic bio companies that I've seen that have been most interesting are the ones that find a way to like make the cycle time super fast um and that that benefits like an AI That's giving you a lot of good ideas but you've still got to test them which is where things are right now um I'm a huge believer for startups that like the thing you want is low costs and fast cycle times and if you have those you can then compete as a startup against the big incumbents”
Language models will pose a serious challenge to Google's search product for the first time, with people soon attempting trillion-dollar business opportunities previously thought impossible
“I don't think we've yet seen the kind of like people go after the like you know trillion dollar like take on Google's um and I think that's about to happen like maybe it'll be successful maybe Google will do it themselves but like I would guess that with the quality of of language models we'll see in the coming years um you know there will be like a serious challenge to Google for the first time for for a search product”
AlphaFold and similar science-dedicated AI products are already adding huge amounts of value and will proliferate, making bio company startups exceptionally promising right now
“there are these science dedicated products whatever like Alpha fold and those are adding huge amounts of value and you're gonna see in this like like way more and way more I like I think I if I were like you know had time to do something else I would be so excited to like go after a bio company right now”
Alignment at small scale has been achieved better than expected, as evidenced by OpenAI aligning its largest models better than previously thought possible, though there are no clear solutions for scaling alignment to AGI over 100 years
“we have some ideas about how to solve the alignment problem at small scale um and we've you know been able to align open ai's biggest models better than we thought we would at this point so that's good um we have some ideas about what to do next um but we cannot honestly like look anyone in the eye and say we see at 100 years how we're going to solve this problem”
AI becoming a mega-buzzword is historically a bad sign for field health, often preceding industry disruption or consolidation
“I like one of the unfortunate things that's happened is uh you know AI has become like the Mega buzzword um which is usually a really bad sign I hope I hope it doesn't mean like the field is about to fall apart um but historically that's like a very bad sign for you know new startup creation or whatever”
Global compute chip supply is insufficient for everyone to train large language models; this constraint alone ensures base model training consolidates to a small number of players
“the world also just doesn't make enough chips so Sam has a work thing he needs to get to”
With the quality of language models expected in coming years, there will be a serious challenge to Google's search dominance for the first time, potentially from new entrants rather than Google itself.
“I would guess that with the quality of of language models we'll see in the coming years um you know there will be like a serious challenge to Google for the first time for for a search product”
Language models will advance much further than people think because significant algorithmic progress remains to come, despite concerns about running out of compute and data
“I think language models are going to go just much much further than people think and we're like very excited to see what happens there um I think it's like what a lot of people say about you know running out of compute running out of data like that's all true but I think there's so much algorithmic progress to come that that we're going to have like a very exciting time”
Historical precedent shows that seismic economic shifts from changing cost structures are consistently underestimated in their impact
“there's like seismic shifts that happen when like the whole cost structure of society change which happened many times before um like the Temptation is always to underestimate those”
Multimodal AI systems integrating text, images, and all other data modalities into single unified models will be developed and will unlock new applications, unlike current systems that treat modalities separately.
“another thing is I think we will get true multimodal models working and so you know not just text and images but every modality you'd like in one model able to easily like uh you know fluidly move between things”
AI systems that continuously learn and improve from usage will replace current models frozen at training time, creating systems that get better the more they're used rather than remaining static.
“I think we will have models that continuously learn so like right now if you use GPT whatever it's sort of like stuck in time that it was trained and the more you use it it doesn't get any better and all of that I think we'll get that changed so very excited about all of that”
AI capabilities in reinforcement learning applied to fusion energy are currently inferior to conventional physics-based approaches from expert physicists and represent hype-driven applications rather than genuine breakthroughs.
“we were talking about like are there all these people saying like I'm doing like these you know RL models for Fusion or whatever and as far as we can tell they're all like much worse than what like you know smart physicists to figure it out”
The marginal cost of intelligence and the marginal cost of energy will trend toward zero over the next decade, creating seismic shifts in cost structures across society comparable to historical technological revolutions, and should be assumed to touch almost everything.
“my basic model of the next decade is that uh the cost of intelligence the marginal cost of intelligence and the marginal cost of energy are going to Trend rapidly towards zero like surprisingly far and and those I think are two of the major inputs into the cost of everything else except the cost of things we want to be expensive the status Goods whatever and and I think you have to assume that's going to touch almost everything um because these like seismic shifts that happen when like the whole cost structure of society change which happened many times before um like the Temptation is always to underestimate those”
In the short term, AI tools for human creators (image generation, code assistance, etc.) will enhance rather than replace creative work for most applications, though over a century timescale AI may eventually execute entire creative projects autonomously.
“I think and I think we're seeing this now that tools for creatives that that is going to be like the great application of AI in the short term um people love it it's really helpful uh and I think it is at least in what we're seeing so far um not replacing it is mostly enhancing it's replacing in some cases uh but for the majority of like the kind of work that people in these fields want to be doing it's enhancing and I think we'll see that Trend continue for a long time um eventually yeah it probably is just like you know we look at 100 years okay it can do the whole creative job”
True multimodal models incorporating every modality in one system that can fluidly move between different modalities will be achieved and will unlock massive applications
“another thing is I think we will get true multimodal models working and so you know not just text and images but every modality you'd like in one model able to easily like uh you know fluidly move between things”
To create enduring differentiated businesses on large language model APIs, companies should focus on a 'middle layer' of tuning existing base models with specialized data flywheels rather than training models from scratch, as base model creation will consolidate to a few providers.
“there will be a small handful of like fundamental large models out there that other people build on but right now what happens is you know company makes large language model API other people build on top of it and I think there will be a middle layer that becomes really important where uh I'm like skeptical of all of the startups that are trying to sort of train their own models I don't think that's going to keep going but what I think will happen is there'll be a whole new set of startups that take an existing very large model of the future and tune it”
A small handful of fundamental large models will exist, with a crucial middle layer of startups taking existing models and fine-tuning them for specific domains, creating enduring value through data flywheels rather than base model training
“there will be a small handful of like fundamental large models out there that other people build on but right now what happens is you know company makes large language model API other people build on top of it and I think there will be a middle layer that becomes really important where uh I'm like skeptical of all of the startups that are trying to sort of train their own models I don't think that's going to keep going but what I think will happen is there'll be a whole new set of startups that take an existing very large model of the future and tune it”
Current language models are not yet good enough to make significant impact on life sciences research, though this will change; benchlife appears more useful than general models in most cases
“so I think the currently available models are kind of not good enough to have like made a big impact on the field at least that's what like most like life sciences researchers have told me they've all looked at it and they're like it's a little helpful in some cases um there's been some promising work in genomics but like stuff on a bench top hasn't really impacted it I think that's going to change”
Three core areas of uncertainty in AI futures are wealth distribution, access to AGI systems, and governance—how to collectively decide what AI can and cannot do
“I think like what is the new social contract like how do my guess is that the things that we'll have to figure out are how we think about fairly Distributing wealth um access to AGI systems which will be like kind of the commodity of the realm and governance like how we collectively decide what they can do what they don't do things like that”
AI systems that can conduct self-improving scientific research—discovering new science, testing explanations, and iterating on research process—are the most promising approach to solving hard alignment problems rather than the 'scary' version of editing code and changing optimization algorithms
“like can we automate like can we automate our own jobs as AI developers very first the very first thing we do can that help us like solve the really hard alignment problems that we don't know how to solve like that honestly I think is how it's going to happen um the the scary version of self-improvement like the one from the science fiction books is like you know editing your own code and changing your optimization algorithm and whatever else um but there's a less scary version of self-improvement which is like kind of what humans do which is if we try to go off and like discover new science uh you know that's like we come up with explanations we test them”
The world's chip manufacturing capacity constrains global LLM development such that no significant number of startups can compete on base model training due to physical constraints on available GPUs/TPUs, consolidating development to a few well-funded players.
“the sort of the kind of like core base model I think that's just going to get too too complex and too expensive and the world also just doesn't make enough chips”
OpenAI is running the largest Universal Basic Income experiment in the world with 1.25 to 1.5 years of remaining duration from a 5-year project, designed to explore UBI as a potential response to AI-driven economic disruption.
“we run the largest uh Ubi experiment in the world I don't think that is uh we have a year and a half a year and a quarter left in a five-year project”
Once AI systems reach sufficient capability levels, they can be instructed with high-level principles like 'don't be racist' and will self-correct complex training data correlations and behavioral patterns, rather than requiring explicit specification of all problematic edge cases.
“could we tell the uh the the agent don't be racist right as opposed to trying to figure out all the different things where the weird correlative data that exists on all the training settings everything else may lead to racist outcomes it could actually in fact do a self-cleansing totally once the model gets smart enough that you can that it really understands what racism looks like and how complex that is you can say don't be racist yeah exactly”
Deep biological drives and reward systems will remain unchanged by AI, and humans will continue to value interaction with other people, creation, status competition, and family formation regardless of AI advancement—these are more likely to persist across centuries than contemporary preferences.
“like all of the deep biological things like I think we will still really care about interaction with other people like we'll still have fun and like the reward you know systems of our brain are still going to work the same way like we're still going to have the same like drives to kind of create new things and you know compete for silly status and like you know form families and whatever um so I think the the stuff that people cared about 50 000 years ago is more likely to be the stuff that people care about you know 100 years from now than 100 years ago”
Planning for worst-case AI scenarios is prudent risk management, but emotional affect should remain optimistic and agency-oriented toward achieving beneficial outcomes rather than resigned to catastrophe.
“you gotta you gotta like plan for the worst you certainly like it's not a strategy to say it's all going to be okay but you may as well like emotionally feel like we're going to get to the Great future and we'll play as hard as you can to get there”
AGI is best defined as equivalent to a median human that could work as a remote co-worker, capable of learning to perform any task a median human can learn to do, with the meta-skill of learning how to learn being the key capability
“AGI is basically the equivalent of a median human that you could like you know hire as a co-worker um so and then they could like say do anything that you'd be happy with a remote co-worker doing like just behind a computer which includes like you know learning how to go be a doctor learn how to go be a very competent coder like there's a lot of stuff that a media human is capable of getting good at and I think one of the skills of an AGI is not any particular Milestone but the The Meta skill of learning to figure things out”
Superintelligence is defined as AI smarter than all of humanity combined in aggregate.
“Super intelligence is when it's like smarter than all of humanity put together”
Language model capabilities will continue to advance significantly due to substantial remaining algorithmic progress, not because of continued growth in available data or compute—data and compute constraints are real but algorithmic innovation will compound these gains.
“I think it's like what a lot of people say about you know running out of compute running out of data like that's all true but I think there's so much algorithmic progress to come that that we're going to have like a very exciting time”
OpenAI runs the largest universal basic income experiment in the world, currently 1.5 to 1.25 years into a five-year project to research AI's economic impacts
“so we run the largest uh Ubi experiment in the world I don't think that is uh we have a year and a half a year and a quarter left in a five-year project I don't think that's like the only solution but I think it's a great thing to to be doing um and you know I think like we should have like 10 more things like that that we try”
While general communication ability becomes more important than specialized prompt engineering, domain expertise and creative vision remain valuable for getting maximal results from AI tools—artists still produce better images from Dolly because they have visual thinking and iteration skills, not because they know magic prompt words.
“I just hope it's not like figuring out to like hack the prompt by adding one magic word to the end that like changes everything else I I like what will matter is like the quality of ideas and the understanding of what you want so the artist will still do the best with image generation but not because they figured out to like add this one magic word at the end of it because they were just able to like articulate it with a creative eye”
The approach to making AI predictions should focus on areas where scaling laws are well-understood or foundational research has already demonstrated capability, rather than extrapolating from buzzword adoption where many initiatives will fail (example: AI + fusion), as the latter indicates an approaching bust cycle.
“I think it's like we like to make predictions where we can be on the frontier understand predictably what the scaling laws look like or already have done the research where we can say all right this new thing is going to work and make predictions out from that way and that's sort of like how we try to run open AI um which is you know do the next thing in front of us when we have high confidence and Kate take 10 of the company to just totally go off and explore which has led to huge wins”
Startups that create specialized versions of large models through tuning with unique data flywheels will develop enduring value because they own domain-specific capabilities not available to others, beyond what can be achieved through prompt engineering alone.
“those companies will create a lot of enduring value because they will have like a special version of they won't have to have created the base model but they will have created something they can use just for themselves or share with others that has this unique data flywheel going that sort of improves over time”
Science-dedicated AI products like AlphaFold are adding 'huge amounts of value' but the more consequential scientific contribution from AI will come from general productivity tools that make individual researchers 2x more productive by helping with code and research direction ideation.
“there are these science dedicated products whatever like Alpha fold and those are adding huge amounts of value and you're gonna see in this like way more and way more I like I think I if I were like you know had time to do something else I would be so excited to like go after a bio company right now like I think you can just do amazing things there um the anyway but there's like another thing that's happening which is like tools that just make us all much more productive uh that help us think of new research directions that sort of write a bunch of our code so you know we can be twice as productive and that impact on like the net output of one engineer a scientist I think will be the surprising way that AI contributes to science”
While marginal costs of intelligence and energy trend toward zero, actors willing to spend vastly more capital in absolute dollars will achieve disproportionate capability gains, creating winner-take-most dynamics where spending curves intersect cost curves.
“but one of the things that is important is it's not like the thing Trends either Trends all the way to zero they just Trend towards there and so it's like someone will still be willing to spend a huge amount of money on compute and energy they will just get like unimaginable amount of intelligence energy they'll just get unimaginable amounts about that and so like who's going to do that and where is it going to get the weirdest not because the cost comes way down but the amount spent actually goes way up”
Future pharmaceuticals and biotech companies enabled by AI could be '100x better' than current incumbents, creating hundred-billion to trillion-dollar companies in rare domains where AI fundamentally changes the development process.
“I think that's going to change and I think uh this is one of these areas where there will be these like you know new 100 billion to trillion dollar companies started those those areas are rare but like when you can really change the way that if you can really make like a you know future Pharma company that is just hundreds of times better than what's out there today that's going to be really different”
Biological simulation and modeling systems are currently poor enough that AI-focused biotech startups should prioritize improving them, as better simulators would unlock substantially more AI-driven discovery.
“the simulators are still so bad and if I were an a if I were a bio means AI startup I would certainly try to work on that somehow”
People worried about finding fulfillment in an AI-driven world are overestimating the challenge because humans have historically solved problems of purpose well; however, the governance challenges of wealth distribution and access to AGI are substantially harder and genuinely unprecedented.
“I'm optimistic that people will figure out how to spend their time and be very fulfilled I think people worry about that in a little bit of a silly way I'm sure what people do will be very different but we always solve this problem um but I do think like the concept of wealth and access and governance those are all going to change and how how we address those will will be huge”
The fundamental difference between base model startups and specialized tuning startups is that the latter train on 1% of total compute/data but that 1% is what matters most for domain-specific performance, creating differentiation through specialization rather than scale.
“they will do the one percent of training that really matters for for whatever this use case is going to be those startups I think they will be hugely successful and very differentiated startups”
Startups should not compete by training their own large language models from scratch due to massive compute and data requirements, but should instead focus on tuning existing base models with domain-specific data and expertise.
“I'm like skeptical of all of the startups that are trying to sort of train their own models I don't think that's going to keep going but what I think will happen is there'll be a whole new set of startups that take an existing very large model of the future and tune it”
Agent systems that use computers and programs through natural language interfaces, allowing iterative refinement through dialogue, will become a massive trend with very large businesses built on this interface
“this idea of like a language interface um where you know you say a natural language what you want in this kind of like dialogue back and forth you can iterate and refine it and the computer just does it for you you see some of this uh with like Dolly and co-pilot in very early ways but I think this is going to be a massive Trend and you know very large businesses will get built with this as the interface”
AI tools that increase scientist and engineer productivity—allowing them to think of new research directions, write more code, and be twice as productive—will be the surprising way AI contributes to science outside of obvious domain-specific models
“there's like another thing that's happening which is like tools that just make us all much more productive uh that help us think of new research directions that sort of write a bunch of our code so you know we can be twice as productive and that impact on like the net output of one engineer a scientist I think will be the surprising way that AI contributes to science that is like outside of the obvious models”
Once AI is smart enough to understand complex concepts like racism, it can be instructed with high-level principles like 'don't be racist' which the system can then apply consistently across its outputs rather than requiring specification of every individual racist correlation
“could we tell the uh the the agent don't be racist right as opposed to trying to figure out all the different things where the weird correlative data that exists on all the training settings everything else may lead to racist outcomes it could actually in fact do a self-cleansing totally once the model gets smart enough that you can that it really understands what racism looks like and how complex that is you can say don't be racist”
The appropriate way to make AI predictions is to identify areas where you can understand scaling laws predictably or where research has already validated an approach, not to apply AI to 'everything'
“I think it's like we like to make predictions where we can be on the frontier understand predictably what the scaling laws look like or already have done the research where we can say all right this new thing is going to work and make predictions out from that way”
Biology has rate limits determined by the speed biological processes run at; synthetic biology companies that find ways to dramatically reduce cycle times can compete against incumbents by combining AI-generated ideas with fast iteration
“there still will be like the rate limit of like bio has to run at its own thing and human Trials take however long they take and that's so I think an interesting cut of this is like where can you avoid that like where are the the synthetic bio companies that I've seen that have been most interesting are the ones that find a way to like make the cycle time super fast um and that that benefits like an AI That's giving you a lot of good ideas”
Quality of ideas and understanding of what you want matters more than prompt engineering tricks; artists will still outperform others at image generation not because of magic words but because of superior visual thinking and iteration ability
“I just hope it's not like figuring out to like hack the prompt by adding one magic word to the end that like changes everything else I I like what will matter is like the quality of ideas and the understanding of what you want so the artist will still do the best with image generation but not because they figured out to like add this one magic word at the end of it because they were just able to like articulate it with a creative eye that you know I don't have”
The Transformer architecture, while remarkably successful, will likely be superseded by fundamentally different paradigms within five years, and it's important to maintain research focus on discovering the next architectural breakthrough rather than optimizing within Transformers.
“I feel bad to say this like I I doubt we'll still be using the Transformers in five years I hope we're not I hope we find something way better but the transform has obviously been remarkable”
Current GPT-3-scale language models are not yet impactful enough to significantly change life sciences research, though promising work in genomics exists and bench-top applications remain limited.
“the currently available models are kind of not good enough to have like made a big impact on the field at least that's what like most like life sciences researchers have told me they've all looked at it and they're like it's a little helpful in some cases um there's been some promising work in genomics but like stuff on a bench top hasn't really impacted it”
The marginal cost of intelligence and the marginal cost of energy will both trend rapidly toward zero over the next decade, fundamentally reshaping the cost structure of everything except status goods
“my basic model of the next decade is that uh the cost of intelligence the marginal cost of intelligence and the marginal cost of energy are going to Trend rapidly towards zero like surprisingly far and and those I think are two of the major inputs into the cost of everything else except the cost of things we want to be expensive the status Goods whatever”
There is no consensus answer within OpenAI about whether it makes sense to have children given AGI, with some people reconsidering family plans entirely and others seeing child-rearing as their primary future occupation
“there's I think no consensus answer to this um there are people who say yeah I'm not I was gonna I thought I always thought I was gonna have kids and now I'm not going to because of AGI like there's just for all the obvious reasons and I think some less obvious ones there's people who say like well it's going to be the only thing for me to do in you know 15 20 years so of course I'm going to have a big family”
Large language models represent a genuine new technological platform comparable to mobile in significance, historically followed by explosive creation of new companies, and this pattern will repeat.
“these very powerful models will will be one of the genuine new technological platforms which we haven't really had since mobile and there's always like an explosion of new companies right after”
OpenAI's strategy combines high-confidence research on the next thing with allocating 10% of the company to totally exploratory research to find new paradigms, yielding 'huge wins'
“and that's sort of like how we try to run open AI um which is you know do the next thing in front of us when we have high confidence and Kate take 10 of the company to just totally go off and explore which has led to huge wins”
The speaker finds it depressing when people decline to have children due to AGI, and argues instead for emotional optimism about positive futures while planning to mitigate risks
“I think like as always it is a a personal decision I get very depressed when people are like I'm not having kids because of AGI uh the EA Community is like I'm not doing that because they're all going to die they're kind of like a techno optimists are like well it's just like you know I want to like merge into the AGI and go off exploring the universe and it's going to be so wonderful and you know just I want total freedom but I think like all of those I find quite depressing um I think having a lot of kids is great I you know want to do that now more than I did even more than I did when I was younger and I I'm excited for it”
AGI is best defined as a system with the capabilities of a median human remote co-worker—capable of learning how to perform any task that such a worker could accomplish, including highly specialized domains like medicine or coding, with the meta-skill of learning being the defining capability.
“AGI is basically the equivalent of a median human that you could like you know hire as a co-worker um so and then they could like say do anything that you'd be happy with a remote co-worker doing like just behind a computer which includes like you know learning how to go be a doctor learn how to go be a very competent coder like there's a lot of stuff that a media human is capable of getting good at and I think one of the skills of an AGI is not any particular Milestone but the The Meta skill of learning to figure things out and that it can go decide to get good at whatever you need”
Claims about AI applied to fusion research (using reinforcement learning models) appear to be much worse than what competent physicists can achieve through traditional methods
“we were talking about like are there all these people saying like I'm doing like these you know RL models for Fusion or whatever and as far as we can tell they're all like much worse than what like you know smart physicists to figure it out”
Research progress into new AI paradigms beyond transformers will continue alongside scaling existing models, making this a 'genuine technological Revolution' even if multimodal continuous-learning models were the only other advancement
“but I think we're likely to keep making research progress into new paradigms as well um we've been like pleasantly surprised on the upside about what seems to be happening and I think uh you know all these questions about like new knowledge generation how do we really Advance Humanity uh I think there will be systems that can help us with that”
Models that continuously learn and improve from use, rather than remaining static from their training date as current models do, will be developed
“I think we will have models that continuously learn so like right now if you use GPT whatever it's sort of like stuck in time that it was trained and the more you use it it doesn't get any better and all of that I think we'll get that changed”
Rather than UBI being the only solution, multiple approaches should be tried (ten or more) to explore different policy responses to AI-driven economic disruption
“but I think it's a great thing to to be doing um and you know I think like we should have like 10 more things like that that we try”
Prompt engineering as currently understood will not exist in five years, replaced by natural language interfaces where users simply describe what they want in language or voice and the system executes it
“I don't think we'll still be doing prompt Engineering in five years I think it'll just be like you and this will be integrated everywhere but you will just like you know either with text or voice depending on the context you you will just like interface in language and get the computer to do whatever you want”
The metaverse and AI are distinct technologies that may independently succeed; in an upside scenario, the metaverse functions more like iPhone-scale paradigm shift in computing interfaces, while AI represents a foundational technological revolution, making AI's impact framework more consequential than asking how AI fits into the metaverse.
“I think it's more like how the metaverse is going to fit into this like new world of AI then AI fit into the metaverse but low confidence the TBD”
Prompt engineering as a specialized skill will become obsolete within five years as natural language interfaces improve, making the ability to communicate clearly more important than learning specific prompt-hacking techniques.
“I don't think we'll still be doing prompt Engineering in five years I think it'll just be like you and this will be integrated everywhere but you will just like you know either with text or voice depending on the context you you will just like interface in language”
Super-intelligence is defined as intelligence superior to all of humanity combined
“and then Super intelligence is when it's like smarter than all of humanity put together”
Arthur C. Clarke's 'Childhood's End' depicts aliens arriving to save humanity but taking their children, with ascension into the overmind being the somewhat more optimistic resolution than the literal narrative
“one of my favorite sci-fi books is called childhood's End by Arthur Clark from like the 60s I think and the I guess the one sentence summary is aliens come to the Earth try to save us and they just take our kids and leave everything else so you know there's a slightly more optimistic than that but yes I mean there's Ascension into the over mind is is is meant to be more utopian but yes okay uh you may not read it that way but yes”
Large language model startups cannot realistically differentiate from each other based on data access alone because internet data is publicly available to all competitors, so differentiation must come from other sources like architecture, infrastructure, or specialization.
“what I have heard and which might be wrong is that large language models depend on data and compute and any startup can access to the same amount of data because it's just like internet data and compute like different companies might have different compute”
Emotionally approaching AI futures from fear and despair rather than optimism and agency would undermine the effort required to achieve beneficial outcomes, and historically fear-based approaches have not delivered progress.
“if we acted from a place of fear and paranoia we would not be where we are today”
Star Trek represents one of the better utopian science fiction universes, particularly those focused on exploring and understanding the universe
“um Star Trek is pretty good honestly uh like I do like all of the ones that are sort of like you know we turn our Focus to like exploring and understanding the universe as much as we can”
Different people hold radically different perspectives on having children in light of AGI development, ranging from pessimism about child welfare in a transformed world, to viewing child-rearing as the primary source of fulfillment in a high-AI economy, but no consensus position exists within organizations like OpenAI.
“this is like a question that comes up at open AI a lot like how do I think about you know how should one think about having kids there's I think no consensus answer to this um there are people who say yeah I'm not I was gonna I thought I always thought I was gonna have kids and now I'm not going to because of AGI like there's just for all the obvious reasons and I think some less obvious ones there's people who say like well it's going to be the only thing for me to do in you know 15 20 years so of course I'm going to have a big family”
Speaker personally believes having children is positive and wants to have a large family, expressing that depression about AI-driven future childlessness (whether from effective altruism concerns or techno-optimist merger fantasies) misses important human values.
“I think like as always it is a a personal decision I get very depressed when people are like I'm not having kids because of AGI uh the EA Community is like I'm not doing that because they're all going to die they're kind of like a techno optimists are like well it's just like you know I want to like merge into the AGI and go off exploring the universe and it's going to be so wonderful and you know just I want total freedom but I think like all of those I find quite depressing um I think having a lot of kids is great I you know want to do that now more than I did even more than I did when I was younger and I I'm excited for it”
Baseline applications of language models in metaverse contexts include computing, simulation, entertainment, education, and AI tutoring, but the speaker hasn't identified additional non-obvious applications despite significant AI potential.
“Computing simulation environments Asians possibly possibly entertainment certainly education right um you know like an AI tutor and so forth those those would be Baseline”
The metaverse is independently cool but not fundamentally dependent on AI; AI's impact on computing, simulation, entertainment, and education is more significant than metaverse impact on AI
“I think they're like both independently cool things it's not like totally clear to me yeah other than like how AI will impact all Computing yeah well obviously Computing simulation environments Asians possibly possibly entertainment certainly education right um you know like an AI tutor and so forth those those would be Baseline”
One cannot identify any single perfect sci-fi universe, but the collective optimistic corner of science fiction is exciting and limited in scope
“I think science fiction is like there's not like one there's not like one sci-fi universe that I could point to and say I think all of this is great but like the collective optimistic corner of sci-fi which is like a smallish Corner um I'm excited about”
The author recently spent time writing a science fiction story about optimistic AI futures and is now seeking recommendations for positive sci-fi to read
“I took a few days off to write a Sci-Fi story and I had so much fun doing it just about sort of like the optimistic case of AGI um that it made me want to go like read a bunch more so I'm looking for recommendations of more to read now”