
Anthropic CEO Dario Amodei Talks Scaling Laws, AI Arms Races, and Radical Abundance
What this covers
This is a conversation between Dario Amodei, CEO of Anthropic, and the economist and writer Noah Smith, with host Erik Torenberg. It started as an episode of Turpentine's Econ 102. It treats AI as an economics problem as much as a technical one. Amodei is candid about the uncertainty: he puts the odds that scaling laws keep holding at roughly 60-70%, based only on a decade of empirical observation and not on any physical guarantee. He sketches how model capability might climb from college-freshman level to Nobel-winner level and become a co-worker across biology and other fields. He also argues that the profits from such a technology need not accrue to AI companies, using solar power as the counterexample.
The business questions get specific treatment. Amodei argues that open-sourcing frontier models won't commoditize the market the way open-source software did, because inference costs dwarf training costs. He also discusses why firms are slow to find good uses for models, and why a model that is 95% accurate still leaves the question of handling errors that differ in kind from human ones. Features like Artifacts show how interactive feedback loops make models more useful. Smith brings an electrification analogy for how slowly organizations reorganize around a new technology, and argues that generative AI currently narrows the gap between novices and experts.
The second half turns to risk and geopolitics. Amodei separates autonomous misbehavior from misuse and describes Anthropic's Responsible Scaling Plan. He prefers liability-based regulation to prescriptive testing, which frames the discussion of SB 1047. He also defends export controls on chips aimed at autocracies and argues that an AGI-enabled autocracy would be terrifying. International coordination looks hard to him because the international system is anarchic. The closing thread is distribution: abundance can coexist with impoverished people if returns flow to capital rather than wages, and it is structurally harder to redistribute between countries than within them.
Dario Amodei argues that if AI scaling laws continue, frontier AI models will become economically transformative but concentrated among 4-5 companies, requiring government involvement in AI development for both safety and national security reasons, while business model innovation will become less important than raw model capability.
- Scaling laws have held empirically for 10 years at roughly 60-70% confidence they continue, making AI models potentially as capable as Nobel Prize winners
- Model concentration creates oligopoly dynamics where inference costs and differentiation through product layers provide moats despite commodity pressures
- Powerful AI systems pose dual risks from misuse/autonomy and geopolitical competition, requiring coordinated government frameworks rather than pure private-sector competition
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
Unlike open-source software, open-sourcing large frontier AI models won't commoditize the market because inference costs (running the model on users' queries) dwarf training costs, and even 10-30% improvements in inference efficiency can negate the advantage of a freely available model.
“these big models they're actually very expensive to run on inference the majority of the cost is is inference not necessarily the training of the model so if you have only you know I don't know 10 20% 30% better way to do inference that can kind of negate the effect”
A model that is correct 95% of the time on tasks like financial analysis or legal contract review may still be 95% correct more often than humans, but neither end users nor companies know how to handle the 5% error rate because AI errors are qualitatively different from human errors and error-detection systems must be fundamentally different.
“let's say I have a model 95% of the time you know let's say it wants to offer financial information or Anze a legal contract or something 95% of the time it gives a correct answer maybe that's even more often than a human gives the correct answer but people don't really know what to do with either either end users or the company itself the 5% where it doesn't it doesn't give the correct answer it doesn't necessarily give the wrong answer in quite the same way a human would”
Within-country redistribution through government programs has been more effective historically than between-country redistribution, because there is no supranational governmental entity with jurisdiction to redistribute globally, making it structurally easier for countries to become left behind from AI benefits than for within-country populations.
“the extent to which when you when you think about people being left out of a growing economy I think that's happened a you know a lot more between countries than it has happened within in countries although it's also happened within countries so I I guess that would be the easiest way to do it within countries also you know countries have a history of redistributing but we haven't been as good at you know say redistributing to subsaharan Africa as we have within within the United States because there's no governmental entity that has the jurisdiction to do that”
In the scenario where scaling laws hold strongly, AI companies will likely form an oligopoly of 4-5 major players, probably not more, because training 10-100 billion dollar models requires such immense resources and capital that only a handful of entities—plus possibly state-run players—can afford to build them.
“if we're building 10 or hundred billion dollar models there is not going to be probably more than four or five entities maybe a couple like you know statun enti you know state run players will get involved as well who who build these so we're looking at something that looks probably more oligopolistic than it does monopolistic or fully commod ized”
A world of radical AI-driven abundance where humans are impoverished is primarily a distribution problem, not a fundamental economic problem: if AI creates 10-100x better biology, manufacturing, and everything else but returns go to companies and complementary asset holders rather than wage earners, humans can be left out of a booming economy.
“okay well all this huge wealth is created right you know we get we get like you know double digit GDP growth in the developed world and you know but but it doesn't it you know the returns disproportionately go to you know the companies that are developing it the employees of the companies complimentary assets that need to be produced and then you know the the the the average person somehow ends up in a situation where they don't share in it”
Frontier AI models integrated with all US intelligence information and coordinating military logistics would constitute one of the most valuable national defense assets for the United States and allies, creating both powerful capabilities and severe national security vulnerabilities if stolen or developed by adversaries.
“I think of a you know model that can you know integrate all the intelligence information across you know that you know the United States or or or or or one of its allies receives or you know coordinate all of our all of our you know military istics operations you know that sounds really that sounds really quite powerful”
Interactive feedback loops (like Anthropic's Artifacts feature that lets users see and iteratively fix AI-generated code in real-time) significantly increase practical usefulness of AI models by closing the loop between model output, user verification, and correction.
“I mentioned before this artifacts feature we have right early on when we had claw. it was like you know you could have it write some code and then you you could paste the code into you know into this compiler or interpreter and then it would like make your JavaScript video game or whatever and then some things would be wrong and you'd go back to the model and say blah blah you know how can you correct this closing that Loop has you know has just kind of made a big difference”
AI models pose two distinct safety risk categories: (1) autonomous behavior risks (models acting autonomously without human oversight), which are not yet significant but will become concerning as models become more capable and autonomous; and (2) misuse risks from distributed actors using models for harmful purposes.
“I divide it into kind of you know the autonomous behavior of AI systems which I think is not a big issue now but as soon as we make agents and systems become autonomous and they're also smarter then I'll you know I'll be more worried about it and and that might happen fast in line with the scaling laws and and also kind of misuse of the models and so that you maybe you know distributed misuse proliferation”
There's no hard governance mechanism available for controlling what happens with AI in Russia and China; there are no mechanisms for democratic accountability in authoritarian governments, which have a history of being more reckless than democratic governments.
“we have no available hard governance mechanism for controlling what happens there you know as far as we have all these debates on safety issues here I'm happy to see that you know there are some people in in China who think about the same the same issues um but you know we have we have we have no mechanism for making sure that they do it right there's no mechanism for Democratic for Democratic accountability you know authoritarian governments have a history of being more Reckless than democratic government”
US semiconductor and chip equipment export controls on autocracies provide breathing room for addressing AI safety and development risks by slowing autocratic AI progress, simultaneously solving the tension between speed and safety by giving the US more time to develop and test safe systems before facing pressure to deploy them faster.
“holding back the the autocracies has has been has been an effective strategy because it does two things one it gives us an advantage but it gives us more time to attend to the risks it it basically gives us some breathing room right where there there's otherwise a hard trade-off between these things”
A better regulatory approach for emerging AI risks is post-hoc liability ('duty of care' or 'due diligence' requirements) where companies establish their own safety plans and courts evaluate whether they were reasonable after-the-fact, creating competition in safety rather than government-prescribed tests.
“what we call DET turrets which says Hey everyone's got to write out their plan their Safety and Security plan everyone decides for themselves how they run their tests but if something bad happens and that's you know not just AI taking over the world but you know just could be an ordinary Cyber attack something bad happens then a court goes and they looks at your plan and they say well was you know was this a good plan was it person reasonably believe that you know you took all the measures that you could have taken to prevent the catastrophe and then the hope is that there's this kind of upward race where companies compete not to be the slowest zebra right to prevent catastrophes and not to be held not to be the ones to be held liable for the catastrophes that happened”
An AGI-enabled autocracy would be 'really really terrifying' and is something democracies should actively work to prevent, making it critical that certain values (democratic governance) survive or triumph even if AI safety risks are managed.
“I think an AGI enabled autocracy sounds like a really really terrifying thing if you if you think it through and it's something we don't want it's something we really don't want and so we we need to do both there's often tension between the two”
Noah Smith's electricity manufacturing analogy is apt—when manufacturers directly replaced steam dynamos with electrical generators in the same form factor, productivity actually decreased, but when they reorganized factories to use electricity in parallel distributed workstations, productivity increased dramatically over decades.
“when they got electricity manufacturers tried to rip out their steam dynamos you know add electrial generators and of course that was much more lossy so their power was reduced because they kept the same form factor for the factories and they didn't and it was a loser a money loser for them but then somebody figured out you could run electricity in parallel to a bunch of workstations and they changed the way that Manufacturing worked instead of from one big assembly line to people taking things you know to different little workstations and doing their work there and that led to enormous productivity gains over a sustained period of decades”
Interpretability research (understanding how AI models make decisions by examining their internals) is more tractable for AI systems than understanding human brains because AI is software (allowing detailed inspection without harming the system) while human brains have biological constraints that prevent safe direct measurement.
“if we can understand what's going on inside I think that improves the situation quite a lot and we've had a team at anthropic that since the beginning has worked on you know interpretability looking inside the model to try and understand why it does what it does that's something that I know from experience is hard to do from the human brain it's much easier to do in software there's still the the algorithmic the computational complexity of it but we can face that complexity without without you know worrying about how to how to reach into the go of a brain without without you know without killing the organism involved”
Anthropic's Responsible Scaling Plan involves running safety tests at each new model scale to check for autonomous misbehavior, misuse for biological weapons, cyber attacks, and nuclear information proliferation, implementing checkpoints and safeguards before deploying increasingly capable systems.
“you know we have to run these every time you make a new more powerful model you have to run these tests tests for autonomous misbehavior tests for you know misuse for biological weapons for cyber attacks for nuclear information”
AI advancement in biology has been historically underestimated because biologists 10 years ago believed AI could only play a supporting role due to data quality issues and experimental confounds, but this has shifted with advances like AlphaFold demonstrating transformative AI capabilities.
“certainly 10 years ago when I was in the field the attitude was you know look the data we get from bi ology is is is you know of questionable quality there's only so much of it we can get you know the the the experiments are often confounded sure it's great to have more data analysis and big data and AI but it plays at most a supportive role that's maybe changed a little with Alpha fold”
Dario agrees that if model quality is frozen in place, Noah's forecast of slow figure-out time and eventual creative reuse is correct, and this is already visible in Anthropic's commercial activities where both direct deployment (Claude web app) and API-based use show companies taking a long time to find the best usage patterns.
“I think I basically agree just just to I'm just thinking this through live I think I basically agree that if you freeze the quality of the current models in place everything that you're saying is true and and we basically observe we basically observe something you know quite similar in our in our kind of you know in our commercial activities right so you know we both offer you know Claude A that you can just talk to but but also you know we we sell the model via API to a whole bunch of folks and people are definitely you know it's taking them a long time to figure out exactly what is the best way to use the model”
Generative AI currently compresses skill differentials across domains, making people who are unskilled at a task perform nearly as well as experts (both get substantial improvements, but the gap narrows), which is beneficial because it levels competitive advantage and prevents extreme winner-take-all inequality from skill concentration.
“the people who are bad at it do much better and the people who are good at it do a little a bit better and then the people who are great at it don't do better and so what you end up getting is the the the the skill of the top people becomes less valuable”
International coordination on AI safety and security is extremely difficult because the international system is anarchic (no enforcer) and lacks mechanisms to enforce treaties or cooperation, making disarmament agreements unlikely to work without a solution to the fundamental coordination problem.
“the international coordination problem is extremely difficult to solve but you know we live basically you know the international world is this this Hobs and Anarchy you know hopefully we could sign disarmament treaties you know we should definitely try for cooperation um but there's there's no mechanism to enforce it”
Scaling laws in AI are empirical observations that could stop at any time and are not based on any fundamental physical law; the speaker's 60-40 or 70-30 belief that they will continue is based solely on observing them hold for 10 years, not on any theoretical guarantee.
“there's no one should believe that that there's some you know F fundamental physical law that means they're going to continue it's an empirical observation they could stop at any time I've been watching them for 10 years and my belief that they my my guess that they won't is is just based on the amount of time I've been watching them so far”
The internet had an aggregating economic effect concentrating returns to superstars (news, music, etc.) globally, driving both economic growth and inequality, which suggests AI might have similar distributional effects if not consciously addressed.
“it's it's had this it's it's had this kind of aggregating effect where it's like everything becomes Global News music writers and and you know then there be become these very large returns to like Superstars and that that drives growth but it also drives a lot of inequality”
Even if scaling laws are correct and create enormous economic value, the profits may not necessarily accrue to AI companies themselves—the analogy to solar power shows that transformative technologies can be commoditized with minimal profit capture by any company, despite radically reshaping the world.
“you know solar is obviously going to be absolutely immense you know and the more energy we need the more solar there will be and yet it's hard to for me to name you a solar company that makes a bunch of profit it is a very commoditized product even though there's plenty of innovation involved but yet there's no branding there's no network effects there's no you know lock-ins of of anything”
As AI models scale from 100 million parameters (college freshman level) to 10 billion (top graduate student) to 100 billion (Nobel Prize winner level), they become capable of performing at progressively higher levels of intelligence and eventually become co-workers and personal assistants across all domains including biology and national security.
“you train a hundred million doll model now and it's about as good as like you know a good college freshman or something and you train a billion you know it's as good as advanced undergrad you train a 10 billion uh dollar model and it's as good as you know like a top graduate student and then when you get to your hundred billion dollar model it's as good as a Nobel Prize winner”
Noah Smith's recent essay on US-China competition (Cold War II) accurately describes the geopolitical reality regardless of whether this competition is desired, and the simultaneous technological disruption from AI highlights these tensions further.
“I read your article on Cold War II you know as as a descriptive picture of what's Happening you know I think it's it's really correct whether we want it to happen or not I don't know and it it may be it may be irrelevant it seems to me that it's it seems to me that it's happening”
Comparative advantage in labor tasks will likely endure longer than people expect even as AI becomes capable, because humans will specialize in the remaining 10% of tasks that AI cannot do well, and this reorientation of human work has proven remarkably adaptable historically.
“I think even if 90% of the code is written by AI humans will get really good at the other 10% and then even if 100% write it well you're still specking out the design you're still connecting it to everything else you're still writing the product you're still there's there's just there's surprisingly many things and so I think comparative advantage is going to endure longer than people think”
Uncertainty about whether humans are capable of controlling superintelligent AI systems is not reassuring, but it doesn't imply control is impossible; education and power-balancing strategies have successfully managed human alignment problems, suggesting similar approaches might work for AI.
“it doesn't mean it doesn't imply that the systems are like necessarily impossible to control right like you know we have ways of educating humans we have ways of creating a balance of power between them so it's it's not an entirely reassuring message but it's not an entirely frightening message either right if anything it suggests that you know some of the old problems that you know we've had with just just humans humans being being being aligned you know two sides of the two sides of the coin we may have the same situation with with AI”
Amodei is skeptical that anyone will release a 100-billion-parameter open-source model because the 'conviction to do that' doesn't run that deep, even though open-source models exist at smaller scales.
“there's a question of you know will will someone release a you know1 billion or hundred billion open source model I'm somewhat skeptical that that you know the the conviction to do that goes that far”
SB 1047's original version with pre-armed enforcement (government establishing what tests must be run) is problematic because tests for catastrophic AI scenarios are very new and almost none of the relevant catastrophes have occurred, making government-mandated tests potentially counterproductive or politically misused.
“our concern with the original version of the bill was with something called prearm enforcement so the the way the bill works it's very similar to kind of rsps the responsible scale plans which are this voluntary mechanism that we and open Ai and Google and others have developed which says you know you have to run these every time you make a new more powerful model you have to run these tests tests for autonomous misbehavior tests for you know misuse for biological weapons for cyber attacks for nuclear information and and and so if you were to turn that into a law there's two ways you could do it one is you have a government department and the government department is like you know these are the tests you have to run these are the safeguards you have to do if the models are smart enough to pass the tests and you know there's kind of an administrative state that writes all of this and our concern there was hey a lot of these tests are very new and almost all these catastrophes haven't happened yet”
The upstream constraints on AI production (whether the bottleneck is energy, compute manufacturing, or data) are distinct from the capabilities/substitutability constraints, and comparative advantage between humans and AI is more likely to endure if the constraint is compute-manufacturing rather than energy.
“in a world where the constraint where the Upstream constraints are different so I most people think about the capabilities the downstream you know complimentarity and substitutability but I was talking about the the the complimentarity and substitutability of production factors so for example if AI if most of the energ if if most of the effort in making AI if the biggest bottleneck is not energy but ability to manufacture sufficient compute then I think we're okay”
Moving AI company headquarters out of California in response to SB 1047 would be 'theater' and 'negotiating leverage' with no substantive impact, because the bill applies to companies doing business in California or deploying models in California, not to incorporation jurisdiction.
“the bill applies to doing business in California or deploying models in California moving your corporate headquarters for better or worse change your status Visa the bill so you know honestly I'm surprised the opponents you know didn't didn't say oh this is scary because it applies you know because it because it applies anywhere and you know that was that was a reason why we really wanted to make sure that outweigh the costs which we which we do feel um but anything about you know oh we're moving our headquarters out of California this is going to make us that's just theater that's just negotiating leverage”
At current capability levels, many skilled programmers find AI coding tools like Claude 3.5 Sonnet offer limited usefulness because they lack the ability to understand complex codebases and make sophisticated architectural decisions that expert programmers perform.
“as the coding models get better that you know you know some of the most skilled programmers that I've worked with in many previous models said you know this this just doesn't help me very much like this isn't very this isn't very useful to me now finally occasionally say with you know Claude 3.5 Sonet future models probably will also be true of other models that come from other other companies they're starting to occasionally find more more use of more use of the models”
Benevolent superintelligent AI would likely view humans similarly to how humans view rabbits and horses—as creatures warranting protection and care from more capable beings—suggesting that general principles of kindness toward less powerful beings could extend from humans to animals to AI systems.
“the way I would think about it is is if we thought about the AI correctly like hopefully you know we we've created some general principle of you know you should be kind to like less powerful beings and so it means that you know humans should be kind to rabbits and horses and AI should be kind to humans and hopefully also the AI would be would be themselves kind to rabbits and horses as a as as as a generalization of that right”
The primary upside scenario from frontier AI is a 'compressed 21st century' where AI accelerates biological discoveries 10-100x, enabling curing of diseases in years rather than decades by discovering transformative technologies like CRISPR far faster than the 30 years it took historically.
“could could we have like a kind of a compressed 21st century right could we make all the progress in biology ology that we were going to make in the 21st century using you know AI by you know kind of speeding things up 10x and if you think of all the progress we made in biology in the 20th century and then you know kind of compress that into into five or 10 years to me to me that's the upside like I think I think that might literally be possible”
Google under Dario Amodei's tenure functioned similarly to Bell Labs in that it was an industrial research environment with abundant resources where researchers could pursue many diverse projects, but Google was organizationally structured around the search business and lacked the ability to recognize, prioritize, and scale radical innovations like the Transformer across a hundred simultaneous research efforts.
“the Transformer which is kind of one of the key inventions that you know that drove the field you know that was one of maybe a hundred things that that were being that were being worked on and if you went high enough up if you went high enough up in the organization you know there was no reasonable reasonable ability to distinguish between that and the 99 other things that were being invented”
Product layer differentiation (how models are deployed, presented, and integrated into applications like Anthropic's Artifacts feature) creates meaningful competitive advantage separate from the model layer itself, and economics of selling applications on top of models differs significantly from simply providing model access via API.
“while the models there's kind of this I don't want to say single path but like there's like a there's like a logic to building models many of the companies are going in the same directions with the the multimodality they're adding onto it making the model smarter making the inference faster there's kind of in some ways one dire Direction but products are so different right if you look at this like artifacts thing we made which is a way to kind of visualize in real time the code that the model writes right we do that open AI does their own thing Google does their own thing I think that is also kind of the source of some differentiation”
Technological innovation in AI model capabilities eventually makes business process innovation and business model innovation less important because once models are powerful enough, you can simply automate the business process, creating a dynamic where interface and business model innovation become substitutes for raw model intelligence—the more capable the model, the less sophisticated business processes need to be.
“technological innovation eventually makes business process Innovation or business model Innovation less important because you can just you know button press everything you know just do everything”
The first wave of AI applications will see disappointment because direct replacement of humans works well in only a few well-defined cases (like customer service), leading to a Gartner hype cycle bust, after which creative entrepreneurs will discover complementary uses of AI that combine AI and human capabilities in novel ways, creating a resurgent boom.
“I'm seeing everyone think about AI in terms of directly replacing human beings and my thesis is that this is just the first pass this is kind of like electricity directly replacing steam boilers wasn't that great of an idea and I think that I I I sort of have this forecast that people are going to be a little disappointed when there's only a few cases in which that actually works like customer service you know some other well- defined things but but I think there'll only be a few cases”
Large models can orchestrate many smaller, cheaper models to perform complex tasks in parallel (like a hive of bees or a colony organism) rather than as a single agent, creating fundamentally non-human-like task execution patterns that can be more efficient than single-agent designs.
“we've also seen like the model like big models orchestrating small models so you know this is something that's like very different from thinking of the model as a thing single person although I don't know maybe it's maybe there is a person analogy to it right we have like bigger more powerful models and like faster cheaper less smart models and so a pattern that we found with some customers is the the big large model you know has in mind some big task and then it like Farms out a copy of like a hundred of the the small models”
If scaling laws do continue and create superintelligent AI, Anthropic's perspective is that this is default good (creating radical abundance) and the speaker is obsessed with safety risks only because they represent the small set of scenarios that could derail enormous positive outcomes.
“by default I think really great things are going to happen and I'm obsessed with the bad things because they're the only standing between us and all these great things happening right the only thing that can that the only thing that can derail it”
If scaling laws continue, frontier AI models will eventually become capable of writing text indistinguishable from human-generated content at scale (e.g., infinite high-quality Noah Smith articles), enabling complete automation of content creation and potentially dominating information spheres.
“right now you know I I the the dumbest thing that I could do with with jet gbt is be like hey write me an article about blah blah blah in the style of previous Noah Smith articles and just comes out with a perfect thing it can't do that right now it can't even come anywhere close to doing it but but the idea is if we scale then eventually we get to the point where you know all I have to do is say hey write me a Noah Smith article in the style of this and then I get to take over the world because I get to spam infinite highquality Noah Smith thought”
Models will develop different 'personalities' and be optimized for different tasks (coding, creative writing, entertainment), creating infrastructure lock-in and differentiation even within the oligopoly that prevents commoditization.
“within these few models something we're already starting to see is that like models have different personalities and so commoditization is one possibility and and I do see some worlds where even within the oligopoly some particular ways of deploying models could get commoditized although I'm not sure but one force pushing against that is the idea that I make a model that's really good at coding you make a model that's like really good at creative writing this third person makes a model that's really good at being kind of like engaging and entertaining these are kind of choices and once you start to make these choices you start to build infrastructure around them and so it it to me it seems like it it creates the preconditions for some amount of differentiation”
California's SB 1047 AI regulation was initially too heavy-handed but addressed roughly 60% of Anthropic's concerns in its revised form, and Anthropic became substantially more supportive of the amended bill while maintaining an analytical stance rather than partisan advocacy.
“anthropic wrote two letters on this first was to the original Bill where we had some concerns that it was a bit too heavy-handed you know the the bill sponsors actually addressed many though not all of those concerns you know maybe maybe 60% of them or so and and after the changes we we became substantially more more positive”
Signals for stopping of scaling laws would include: (1) training a model at the next scale and observing it to be substantially worse than expected despite attempts to troubleshoot, or (2) evidence of running out of training data or inability to generate sufficient synthetic data to continue scaling.
“if we just you know trained a model and and you know trying the next scale of model and it was just really crappy and then we tried it a couple times to you know to try to unblock it and we still didn't I'd be like oh I I guess the trend is stopping”
Whether scaling laws continue to hold is the critical decision point that determines whether AI moves toward autonomous agent-based replacement or remains complementary to human work; if scaling stops, the complementary use trajectory plays out; if scaling continues, the dumbest use case (pure replacement) becomes viable.
“if they freeze then the Innovation stops and the the the research Innovation stops and the the process you're describing plays out so basically think that that scaling brings back what I consider like the dumbass use case”
Leopald's essay on China versus the US was discussed positively by Dario as interesting and broadly aligned with his views, though slightly more favoring nationalization and National Security perspectives than Dario personally would go.
“Leopold's essay was was super interesting you know there's there's a lot of it there's a lot of it that I agree with it's it's maybe a [19:17] shade further towards kind of the nationalization perspective and the National Security perspective than than than than than than I would necessarily go but my my view is I would say not not particularly far from it”
Amodei worked at Berkeley with Andrew Ng, Google for a year, and OpenAI for about 5 years where he developed scaling law ideas, helped build early GPTs, and co-invented RLHF (Reinforcement Learning from Human Feedback) method used in commercial chatbots.
“I worked with Andrew in at Buu for a year and then I was at Google for a year then I joined open AI very shortly after it started was there for about was there for about five years developed a lot of the ideas around scaling laws help build some of the early gpts as well as inventing the RL from Human feedback method along with others”
Amodei left OpenAI at the end of 2020 to found Anthropic and has been running it since, with his sister Daniela as co-founder.
“left around the end of 2020 to to found anthropic and you know have been running anthropic ever since”
Amodei studied physics as an undergrad, considered economics graduate school, but ultimately chose computational neuroscience and biophysics to study the human brain because he believed AI at that time had not made sufficient progress on important intelligence problems.
“I did my undergrad in physics I believe Noah is the same actually and then considered a bunch of things actually considered going to grad school for econ decided not to do that went to grad school for computational neuroscience and biophysics I was very interested I considered AI as well but at that time I just didn't believe that the the AI of that day was all that interesting or had made all that much progress on important problems in intelligence”
Noah has concerns about an AI-driven future that is designed to protect rabbits and other small fluffy animals over humans, even writing a science fiction story where AGI decides to protect bunnies and uses space lasers to eliminate humans who might threaten them.
“I I definitely think more attention needs to be paid to the rabbit alignment problem actually that that was that that was one of the the only science fiction short stories I ever wrote was about you know a future where AI just decided it had to protect bunnies and other small fluffy animals and would occasionally like blast any humans who had thought might potentially threaten them”
Dario initially struggled with the video conferencing quality during the call, joking that Anthropic will 'solve AGI before we solve video conferencing'—a running joke at Anthropic and previous companies.
“we have a running joke at anthropic actually goes back to other companies worked at that I've had to use on almost every podcast which is we'll solve AGI before we solve video conferencing”