
Microsoft Goes FULL NUCLEAR to Develop AGI...
What this covers
This is a roughly 19-minute news-style video in which Wes Roth works through several linked stories about Microsoft, with Sam Altman appearing through clips about OpenAI's o1 model. It starts with Constellation Energy's 20-year agreement to supply Microsoft with carbon-free power from Three Mile Island, with a target of 2028. Roth clarifies that the unit being restarted is unit 1, which ran safely until 2019, and not the reactor that failed in the accident. He also covers nuclear fuel's energy density and the point that electricity is a bottleneck for both training and inference.
The middle section turns to Microsoft's Grok Moe model, a mixture of experts design with 16 experts and 6.6 billion activated parameters. Roth cites its results against dense models on AGI Eval and HumanEval, then makes the case for small models on edge devices such as phones, cars, and thermostats. He also claims GPT-4 was the point at which the public came to see frontier models as mixtures of experts, a claim the material marks as contested.
The longest stretch concerns coding. Roth traces HumanEval scores from 32.2% to 99.39% in three years and asks what that does to software work. He separates the programmer who implements from the engineer who plans and oversees, and argues the division of labor will change quickly even without full replacement. Whether experienced engineers can be replaced is left open, and the idea that engineers become overseers is flagged as contested. He also argues that an engineer who masters AI tools will beat an equally skilled one who doesn't.
The last segments use Altman's remarks on o1 as the first "reasoner" in OpenAI's five-level framework (chatbots, reasoners, agents, innovators, organizations), set against the "system one" character of earlier GPT models. Altman expects new non-chat uses of o1 to take months to emerge. The chapter list also includes short Chipotle and T-Mobile segments.
Microsoft's AI investments—including the Grok Moe model and Three Mile Island nuclear restart—demonstrate how AI infrastructure demands are driving technological and energy innovation, while smaller efficient AI models are becoming competitive with massive ones, reshaping software development and enabling new AI reasoning capabilities.
- Mixture of experts models like Grok Moe achieve competitive performance at 1/2 to 1/7 the size of dense models
- AI coding benchmarks have improved from 32% to 99% accuracy in 3 years, shifting the labor division in software engineering
- OpenAI's o1 reasoning model represents a paradigm shift from system-1 thinking to genuine reasoning, unlocking new use cases and deployment models
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
Three Mile Island's worst nuclear accident in US history was caused by the failure of reactor unit 2; unit 1 operated safely until its shutdown in 2019, and it is unit 1 that Constellation Energy is restarting, not the reactor that failed.
“the thing that went Ki was unit reactor 2 so the point is T 2 the second reactor was decomission that's the thing that caused the radioactive gases to be released into the environment and there's TMI 1 and that one operated until it's shut down in 2019 so that one's running just fine”
Nuclear fuel is extremely energy dense: one ton of coal produces the same amount of energy as 120 gallons of oil, which is the same amount of energy as one tiny uranium pellet.
“nuclear fuel is extremely energy dense one ton of coal produce the same amount of energy as 120 Gall of oil and the same amount of energy as one tiny uranium pellet so 1 in tall so it's very energy dense compared to anything else we have”
Grok Moe is a mixture of experts model with 16 experts and 3.8 billion total parameters, with 6.6 billion activated parameters, that outperforms a 7 billion parameter dense model and matches the performance of a 14 billion parameter dense model on the same training data.
“this model that they made it's 16 experts by 3.8 billion parameters so with 6.6 billion activated parameters so the effect of sort of model size this model outperforms a 7 billion model that's you know the dense model the one big chunk model and it matches the performance of a 14 billion dense model trained on the same data”
Energy and electricity are a major bottleneck for AI development, affecting both model training and inference operations.
“energy electricity is seen as a big bottleneck for AI development for training the models for inference to a lesser degree but overall we need a lot of power to run and develop these models”
Tech companies are investing billions in AI coding tools because improving software engineer productivity—either by augmenting engineers or reducing headcount—would be extremely profitable.
“these people spend billions and billions of dollars on various software developers if the AI tools can help them offset some of that cost of course that's very appealing to them even if it doesn't necessarily offset the cost but improves the efficiency it's still extremely extremely appealing it's a very big juicy Target to hit if you have the best AI tool that either a m you know turns most you know software Engineers or whatever word you want to use for that turns them into just Rockstar Engineers whether it does that or or eliminates the need for half of your software Engineers like either way that's a big big deal it would be extremely lucrative to be the owner of that you know AI tool”
Grok Moe shows strong reasoning performance especially on code, math, and logic tasks, making it important for memory-constrained environments and latency-sensitive scenarios.
“it shows strong reasoning especially code math and logic and so right now if you're kind of not aware there's a lot of people that are debating and considering how much will coding change over the next several years as AI keeps getting better and better at coding”
Constellation Energy has agreed to a 20-year contract to supply Microsoft with carbon-free power from Three Mile Island, with plans to bring the reactor online in 2028.
“Microsoft is looking for a lot of energy to run the data centers needed for AI and looks like Constellation Energy the company that's putting the power plan back online looks like it wants to put it back online in 2028 so just under four years from now and when we're talking about Microsoft like doubling down just betting it big so they've agreed to purchase this energy for for 20 years”
Small language models running on edge devices (phones, cars, thermostats) have significant advantages in cost, speed, and accessibility compared to large centralized models.
“smaller models are faster they're cheaper to run they're cheaper to train they have a lot of advantages they can even be installed on edge devices on phones and thermostats and computers in your car because you're going to have enough computer power there to to run it on those Edge devices”
How software development work will be divided between AI and humans—what labor division emerges—will certainly change over time, and change rapidly, even if total replacement does not occur.
“I think that's still up for debate there are people that make very strong arguments why that will not be the case and certainly we haven't seen anything that proves them wrong yet but I think it is obvious now that how we do things will be changing how we approach coding what work will be done by AI what work will be done strictly by humans kind of like what what does that Labor division look like I think for sure that will be changing over time changing rapidly”
The CEO of Constellation Energy framed restarting Three Mile Island as 'the most powerful symbol of the Rebirth of nuclear power as a clean and reliable energy source'.
“the CEO of Constellation Energy saying the decision here is the most powerful symbol of The Rebirth of nuclear power as a clean and reliable energy source”
Mixture of experts models scale more efficiently than dense models, solving training issues specific to mixture of experts architectures through gradient-informed approaches.
“Microsoft says here they're saying thise models they scale more efficiently than dense models so the the big giant models where it's like allinone but there are issues with how we train these models and this is sort of an approach in architecture to make the training better for specifically mixture of experts models”
GPT-4 represents the threshold where people widely recognized that frontier LLMs use mixture of experts architecture, making it a pivotal moment in public understanding of LLM design.
“gp4 was the thing where people realized that it wasn't one giant model it was a actually a collection of of experts a mixture of experts each with its own sort of Areo expertise”
AI tool improvements in coding will shift labor, with software engineers becoming more like overseers and high-level planners rather than writing low-level code, as AI handles the grunt work.
“some people think it might completely replace software Engineers I think most people that are the kind of AI insiders that are running these company Jensen hang the CEO of AWS Amazon's cloud services and and a lot of other people that saying well it's not going to replace them but it it is really going to change in How We Do software development software engineering because AI is going to do a lot of the grunt work and so the engineers will kind of get lifted to more higher tasks more like overseers and planning and kind of putting it all together”
In 5 years, with continued algorithmic improvements and better model architectures, AI models will ace coding benchmarks, but it remains an open question whether this will replace experienced software engineers who excel at planning, visualization, error correction, and teamwork.
“if you fast forward 5 years you know and let's say we have all those sort of improvements improve you know the algorithmic improvements the improvements in model architecture people learning how to use and put these things together you know it's going to be acing these coding benchmarks you know does that mean that it replaces a smart software engineer with experience with you know experience working in teams and kind of that that planning abilities kind of visualizing everything and seeing how everything fits together and then being able to correct errors and stuff like that I don't know I think that's still up for debate”
On the HumanEval coding benchmark, AI models have improved from 32.2% accuracy (pre-September 2021) to 99.39% accuracy (Claude 3.5 Sonnet), a roughly 3x improvement in 3 years.
“before September 2021 the best thing we had looks like had 32.2 accuracy on this human ofal Benchmark so 32 let's say just for Simplicity and then a year later we had code Da Vinci at 65.8 a modif I GPT 4 at 85 and looks like now we have ones that clock in at 99.39% 3 Cloud 3.5 Sonet”
On the AGI Eval benchmark measuring human-level performance, GPT-4 scores 37.6%, Gemini 1.5 Flash scores 45%, GPT-3.5 scores 48%, and Microsoft's Grok Moe (6.6B parameters) scores 48.2%, showing that the tiny Grok Moe model is competitive with much larger models.
“so GPT 40 on that AGI Val it did 37.6% Gemini 1.5 flash did 45 looks like GPT 3.5 did 48 surprisingly did really well but Microsoft's new model the tiny 6.6 billion parameter model did 48.2 right”
Grok Moe, at 6.6 billion parameters, scores 74.4% on the HumanEval benchmark, compared to 65% for a 175 billion parameter model in September 2022, representing roughly 2.7x parameter efficiency.
“so GPT 40 is 90.2 right and so this is the the human of valve Benchmark across many different models the leftmost one is the one we're talking about grin Moe this this tiny model this 6.6 billion model for comparison we believe that gbt 4 the number that they said was 1.7 trillion now we don't know if that's an Apples to Apples comparison because here you know they we know the architecture exactly of GPT 4 or we don't but the point is you know on this human ofal grin Moe is at 74.4 so again in let's say September 2022 the best model for this particular Benchmark right clocked in at 65 and it was 175 billion parameters big this new one is at 74 at less than 7 billion parameters”
GPT-4 was revealed to use a mixture of experts architecture, not a single monolithic model, with each expert handling distinct types of queries.
“gp4 was the thing where people realized that it wasn't one giant model it was a actually a collection of of experts a mixture of experts each with its own sort of Areo expertise so instead of thinking of something like GPT 4 as sort of this one big model instead when you ask it a question it gets routed or gated to the appropriate sort of submodel or expert that expert answers the question”
OpenAI's five-level framework of AI capability progression is: (1) Chatbots, (2) Reasoners (which has just been reached with o1), (3) Agents, (4) Innovators (capable of discovering new scientific information), and (5) Full Organizations.
“we have these five levels of AI we talk about uh the first was chatbots the second which we've just reached now is reasoners uh the third is Agents the fourth is sort of innovators the ability to figure out new scientific information and the fifth is full organizations”
The distinction between programmer and software engineer is meaningful: the former implements, the latter plans, oversees, and orchestrates—these roles may diverge as AI handles more implementation work.
“so this person point out wouldn't this be the rise of the software engineer and the end of the programmer so I mean that sparked a bit of debate which is very interesting of what exactly kind of those roles were will shift to and what are we going to call it coding software engineering Etc but one thing is apparent is a lot of these companies are really really interested in developing these AI tools”
Between two equally talented software engineers with identical knowledge and work ethic, the one who invests time in mastering AI tools will be a better engineer than the one who does not, all else equal.
“if you take two equally talented software Engineers you know all else being equal the one with the stronger AI knowledge and ability and the person that can kind of like manage AI better that's the person that's going to be a better software engineer again all things being equal if you can run a perfect kind of ab test split test right like clone somebody perfectly right they have the same knowledge the same intelligence the the same work ethic right and if one of them took the time to really figure these tools out kind of like stayed ahead of the curve while the other one you know just decided to rewatch the Silicon Valley show during that same time the one that learned to use the AI tools would probably win”
New interaction paradigms and use cases for o1 will emerge over time; usage patterns that are not chat-based interfaces will develop, taking months for both OpenAI and third parties to build and for users to discover.
“I think we're that early with 01 there will be totally new ways to use it that are not just a chat interface uh it'll take us a while to build those and other people a while to build those it'll also take users a while to figure out how to use it and this is pretty different than the GPT models um we have these five levels of AI we talk about uh the first was chatbots the second which we've just reached now is reasoners uh the third is Agents”
The GPT series of models excelled at 'system one' type thinking (quick, pattern-matching responses) but OpenAI wanted to develop systems that could engage in deeper reasoning.
“the GPT series of models was amazing at sort of system one type thinking for lack of a more nuanced word but we knew what we really wanted was systems that could reason and there's so much value if you can have ai that can reason through more complex problems”
O1 is the first system that can perform advanced reasoning on difficult programming challenges, math problems, and science questions, and it will unlock significant new valuable use cases.
“01 really is the first system that can do pretty Advanced reasoning um you know if you give it a difficult programming challenge or a difficult math problem a difficult science thing you need help with uh you can really get pretty extraordinary results and we believe this is going to unlock uh over time this will look as significant as the GPT series and unlock a huge set of new very very valuable use cases”
AI is creating incentives for the resurrection of nuclear power as a viable energy source.
“AI is almost like creating these incentives for us to bring back nuclear power Microsoft also recently announced their brand new model called grin Mo”
The transition from level 1 (Chatbots) to level 2 (Reasoners) took a long time, but level 2 will enable level 3 (Agents) relatively quickly after, and agentic capabilities will be impactful.
“this move from one to two took a while but I think the most exciting one of the most exciting things about two is that it enables level three relatively quickly after and the agentic experiences that we expect this technology to eventually enable I think will be quite impactful”
The improvement curve for new reasoning model paradigms is steep; models that cannot solve problems today will be able to solve them in a few months, and solve even more in a few months after that.
“one of the many fun things about these moments of new paradigms is that the curve the Improvement curve is really Steep and so uh you know things that the model just can't solve right now in a few months they'll be able to solve few months after that be able to solve even more”
O1 represents the GPT-2 stage of reasoning models; over coming years it will reach GPT-4 equivalence in reasoning, with significant improvements visible in coming months as it evolves from o1 preview to o1 full release.
“I think of this as like a we're at the gpt2 stage of these new kind of reasoning models and uh you will see it over the coming years go up to the gp4 equivalent um but even in the coming months you'll see it get a lot better as we move from o1 preview to 01 which we shared uh some metrics for in our launch blog post uh it's a very significant step forward”
CEO of AWS Amazon's cloud services (Jensen Huang) argues that AI will not replace software engineers but will meaningfully change how software development is done.
“Jensen hang the CEO of AWS Amazon's cloud services and and a lot of other people that saying well it's not going to replace them but it it is really going to change in How We Do software development”
Semantic saturation is a psychological phenomenon where repeating a word many times causes it to lose meaning.
“there's this thing called semantic saturation that's like when you say a word so many times that it loses all meaning has that ever happened to you like you say door 50 times and you're like G and just your brain no longer kind of like links that word”
Chipotle is introducing robots (called Ocado) to help prepare bowls, and this is expected to be the first of many robotic devices rolling out across restaurants nationwide.
“Chipotle introduces robots to help make the bowls and they're calling it ocado so expect more and more of these uh robotic devices that are going to be rolling out to the various restaurants across the country I think Chipotle is just the first of its kind”
T-Mobile and OpenAI held a 2-hour fireside chat in which they announced developments and discussed AI strategy, but the event received relatively low attention compared to tech events by Apple, Google, and NVIDIA.
“T-Mobile and open AI joined forces so they had a whole little fireside chat which just did not get too much attention it was like a 2hour long session of them talking about stuff and announcing stuff it was just kind of I mean T-Mobile isn't as exciting to watch what they're talking about as you know like when Apple does an event everybody watches that even Google Nvidia like a lot more eyes on that T-Mobile not so much”
A comment from a YouTube viewer ('internal cold 14') on Wes Roth's software engineering video reframed the title 'The End of Software Engineers' as 'The Rise of the Software Engineer and the End of the Programmer', sparking productive debate about role definitions.
“this person uh internal cold 14 great comment said in response to your thumbnail the end of software Engineers which was that was like the quote that was kind of floating around I used it in a title because that was kind of like the language that was being used and so this person point out wouldn't this be the rise of the software engineer and the end of the programmer”