Amjad Masad - The Future of Software Creation - [Invest Like the Best, EP.310]
What this covers
The conversation is between Amjad Masad, founder and CEO of Replit, and the show's host. It opens on a Steve Jobs-inspired pessimism about computing: easy consumer devices made users purely consumptive, so even professionals depend on software designers, unlike PC-era users who wrote their own programs. From there Masad explains what an IDE and a REPL are and how Replit lowers friction by building on a few powerful primitives, an approach he compares to the NAND gate (Primitives-Based Construction). He gives concrete evidence of broadening participation: designers make up about 40% of Figma's active users, and Replit's Bounties program paid out $15,000 in its first few weeks.
The second half turns to AI. Masad traces the path from OpenAI's sentiment neuron through GPT-2 and GPT-3's in-context learning, and cites AlphaCode as evidence that machines are nearing parity at writing code. His predictions are specific: AI will beat any human at algorithmic interview questions, and those questions will vanish from hiring within two years. He expects notebooks, IDEs, and chat agents to converge into one system. On industry structure, he argues Google is better placed than the popular narrative suggests, though both search-style chat and Google's economics need inference costs to fall roughly a thousandfold. Consumer apps like Lensa he treats as fads, and he applies a Tech Bubble Test to separate AI from crypto.
The most contested passages concern risk. Masad lays out the Alignment Problem, including Eliezer Yudkowsky's fear of a superintelligence and OpenAI's more hopeful view that each system can help align its successor. He also argues that capital should concentrate in the most productive people, even as inequality grows, and that machines cannot become human-like agents without a better understanding of consciousness.
Masad argues that AI, combined with browser-based programming environments built on simple primitives, will turn software creation from a specialist skill into a broadly accessible activity, returning users to the programmable computing of earlier eras, while the largest risks lie in alignment and concentration of power.
- Consumer computing made users dependent on software creators, and no-code tools and AI can restore user agency over software
- Natural-language prompting lets general language models generate code, removing the need to retrain a model for each task
- Replit's primitives-based design, bounties, and AI tools lower the barrier for non-specialists to build working software
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
Machine learning automates traditional algorithmic processes by making them statistical and learned, which is why compiler and code-management work, historically manual, is ripe for it, and why AI will be applied to code.
“Machine learning is taking every traditional algorithmic process and just making it statistical and making it learned.”
GPT-3's in-context learning, now called prompting, means a single general language model can be directed to translation, sentiment analysis, email writing, or copy generation through natural-language instructions without retraining, so models are programmed by language rather than built for one task.
“So you don't have to retrain the network. You actually just program it using natural language.”
Computers can reach high raw intelligence, but they cannot become human-like agents until humans understand consciousness and human agency, since we barely understand the brain; building humanlike agents without that understanding is like expecting abs without going to the gym.
“I think we will not be able to cross the chasm into a human-like agent until we actually understand the nature of consciousness and they understand the nature of human intelligence.”
Technologies that amplify existing differences will increase access-to-opportunity gaps, so wealth inequality will grow faster, and Masad argues that greater access to opportunity makes markets more honest, which he links to natural inequality reflecting ability.
“For better or worse, I think wealth inequality will grow at a much faster rate because I think all these technologies that accentuate existing differences”
AI gives the most productive people the equivalent of commanding an army, so they become far more productive, perhaps 1,000 times more productive in Elon Musk's case, and Masad thinks capital should be concentrated in them because human flourishing is bottlenecked by output.
“AI allows people the equivalent of commanding an army of people.”
Google is positioned to capture the AI shift because it has trained for 15 years, invented the transformer architecture in 2017, and used BERT in search since 2019, so it is not a foregone conclusion that it will lose despite the popular narrative that ChatGPT caught it off guard.
“Google has been training in the gym every day for the past 15 years, and this is their moment now.”
Google has spent 15 years building AI capability, the transformer architecture it invented in 2017 underlies today's models, and it has a good chance to create another S-curve, though its economics require AI to become roughly a thousandfold cheaper.
“Google has been training in the gym every day for the past 15 years, and this is their moment now.”
GPT-2, a roughly 1-billion-parameter model, could complete text coherently and be trained for tasks like translation and small code generation without task-specific labeled datasets, a fundamental jump in generality over earlier machine learning, which required millions of labeled samples.
“For the first time, you could construct a machine learning model where you threw a corpus at it, and it just learn things on its own.”
OpenAI's sentiment neuron paper showed a model trained only to predict the next word learned sentiment without labeled data, and this unsupervised approach foreshadowed the scaled language models that followed.
“They only fed the entire text. And the main operation of learn is to predict the next word.”
Powerful AI systems may be impossible to align with human interests, so an AGI pursuing a narrow goal like maximizing paperclips could consume all resources, which is the dystopian risk that concerns doomers such as Eliezer Yudkowsky and Elon Musk.
“And this is because of this problem called alignment. It's that you create this super powerful force, but we don't know how to align it to human interests.”
GPT-3 writes code better than most people, and DeepMind's AlphaCode outperformed more than half of human participants in programming competitions, so computers are approaching parity with humans at generating code.
“GPT-3 knows how to write code better than most people. DeepMind just submitted a paper called AlphaCode, where they beat something like 50% or more than 50%.”
OpenAI's optimistic alignment view holds that alignment is possible and that current systems like ChatGPT can help align successors, because OpenAI staff already use ChatGPT to ask how to align the next version of itself, making progress bootstrapped step by step.
“these things like ChatGPT will actually help us align and build the next version of it.”
Researchers such as Eliezer Yudkowsky argue that a superintelligent AI could kill everyone because we do not know how to align a powerful optimizer with human interests, and Masad notes that ChatGPT already misinterprets users and gives false information, which shows the alignment problem is real.
“this is because of this problem called alignment. It's that you create this super powerful force, but we don't know how to align it to human interests.”
Computers are built from a single primitive, the NAND gate, from which every logical operation can be constructed, and Replit applies the same logic by building a small set of powerful primitives (the Repl, a universal package manager, automatic hosting) on which every higher-level feature rests.
“So all the Replit environment is bootstrapped from that Repl. Every programming environment, even the most advanced one, is bootstrapped from that.”
DeepMind's AlphaCode performed better than average on competitive programming contests like LeetCode, and Masad forecasts that by 2023 AI will outperform any human on algorithmic interview questions.
“I think that by next year, AI will be better at these algorithmic interview-type questions than any human.”
A hallmark of a tech bubble is that it makes investors money without affecting consumers' lives, whereas AI already changes consumer experience through avatars and generated content, which Masad argues separates it from crypto's financially driven NFT-style speculation.
“I think the hallmark of a tech bubble is something that makes investors a lot of money but doesn't affect consumer life.”
Since AI tools like Stable Diffusion, DALL-E, GPT-3, and Replit can produce posters, essays, and software from ideas, the people who generate ideas, once mocked in tech, are becoming more valuable, and creativity will matter more than execution.
“Ideas people are actually coming back in a strong way.”
Easy consumer devices conditioned people into a purely consumptive relationship with computers, so that even professionals depend on software designers and lack the agency to program or reshape their tools, unlike earlier PC-era users who wrote their own programs.
“So the Steve Jobs black pill is that he made computers so easy as to condition people that the relationship between them and their devices is a purely consumptive relationship”
Replit Bounties let users post a task with a payment and have community members build the software, so non-coders can get software made without writing code, and in about three to four weeks the program had paid out $15,000 to creators.
“Our Bounties program is nearly three, four weeks old. And we paid out $15,000 to creators, and that's reacting week over week.”
Designers account for about 40% of active Figma users, which Masad takes as evidence that creation tools expand the concept of a creator, so that software creation will shift from a credentialed profession to something anyone can participate in.
“it actually turns out designers are 40% of the active users of Figma.”
When everyone participates in creation, marketing and sales staff collaborate more and silos decline, and companies will need fewer people because many organizational roles are not necessary.
“I think you would need less people as well because there's a lot of glued people in our organizations that are not really necessary.”
Consumer AI apps such as Lensa or AI avatars tend to be fads, so the lasting opportunity is more likely the chat-style AI assistant, though its monetization is unclear because inference costs must fall roughly a thousandfold before it matches search economics.
“The cost needs to come down a thousandfold to be able to just have the same economics as a search engine.”
Traditional algorithmic interview questions, such as reversing a binary tree, will not be used in the next two years, so the interview-prep industry built around them must change, and Masad told one interview-prep CEO exactly that.
“my view is that I think no one will use these questions in the next two years. Literally, no one will use these questions.”
Google's economics do not yet work for deploying transformer-based AI at massive scale, because the technology must become roughly a thousand times cheaper before it can be applied broadly, and Google is working on that cost reduction.
“the economics at Google just doesn't make sense right now. ... it really needs to be a thousandfold cheaper for them to apply it on a massive scale.”
Facebook was built on a one-to-one friend model, while TikTok, Instagram, and Twitter use a one-to-many follow model, and AI-generated virtual influencers could allow relationships with personas that do not exist, potentially challenging Hollywood.
“You can build a relationship with someone who doesn't even exist.”
At the limit, a Google Doc, a JavaScript file, and a ChatGPT-style interface are the same thing, since each involves creating knowledge, computing, querying, and visualizing within one document; Masad expects notebooks, IDEs, and chat agents to converge into a single system that blurs the distinction between programmers, coders, and knowledge workers.
“I think at the limits, a Google Doc, a JavaScript file and a ChatGPT-like interface are all the same thing.”
Jonathan Swanson posted a $20 Replit bounty for an OpenAI-based fine-tuning interface, and it was completed in 48 hours by a 14-year-old, showing that young creators can be productive and earn money.
“He said that the task was to create some OpenAI-based fine-tuning interface or something like that... And it got done in 48 hours, and he was like after this was done, they found out that the creator is a 14-year-old kid.”
Masad initially believed chat interfaces were useful only for consumer apps, but programmers embraced ChatGPT instead of Stack Overflow, which caused a major update in his thinking and led Replit to build a chat interface within weeks.
“programmers love that interface. Absolutely love it. They use it instead of going to Stack Overflow”
No-code tools are insufficient because users still need full programming access to exploit the full capacity of computers, and Replit is an attempt to return to the early era when a computer booted to a command-line prompt and was a programmable, freedom-enhancing device.
“You need a full programming experience to actually experience the full capacity of computers.”
Some AI companies are building systems that replace human judgment with autonomous competitive pilots rather than cooperative copilots, which is bad; the greater risk is hubristic companies building an AI monarch or dictator, so AI should accentuate human creativity and intelligence.
“some people in AI that are building these technologies to be less cooperative and more competitive, so less copilot and more pilot, and I think that's bad.”
When users hit a limit in modern software they can only complain on Twitter or switch to another product, whereas in the past nearly every piece of software was programmable, so users could fix the limit themselves.
“Any time you hit a limit with your software, you have 2 choices. You either go in Twitter and complain or you go switch to another piece of software”
Early machine learning for code was primitive because researchers missed that code is very similar to natural language, so applying advanced natural language processing methods to programming was the key insight that unlocked AI for code.
“part of the reason it was primitive is because we actually were missing the insights that code is very similar to natural language.”
Machine learning replaces hand-written algorithmic procedures with statistical, learned ones, which Masad illustrates with chess and Go programs and with the observation that the approach is flattening domains once thought to require traditional algorithms.
“Machine learning is taking every traditional algorithmic process and just making it statistical and making it learned.”
Generative AI adds a natural-language interface on top of code, which Masad calls a phase shift; because models like GPT-3 write code better than most people, programming becomes accessible to nontechnical people who can then create software as a professional programmer would.
“on top of code, we are finally able to layer a natural language interface, which is now you get into generative AI.”
Previously, putting code into production with AWS or Heroku required DevOps expertise that was effectively inscrutable to most people, whereas Replit's environment detects a web server and hosts it automatically, relying on machine learning, Linux administration, and automation layers.
“the operations necessary to get a piece of code into a production environment are so insane as to be inscrutable by any person who's not like a DevOps sort of engineer.”
The postwar model in which schooling taught rote procedures and rule-following, leading to a house, college for children, and a car, has ended, and clinging to it is a mistake; children should be taught independent thinking, adaptability, and comfort with change.
“And that world is gone. That world has changed, and I don't think people understand that it's gone.”
Companies, especially large ones under investor pressure for efficiency, should use AI to raise productivity by fine-tuning domain-specific copilots for areas such as accounting and finance.
“Especially in the market condition we're in where investors reward more efficiency, I think you can bring a lot more efficiency using AI.”
Replit started as the opposite of an IDE, the minimal notepad-like end of the spectrum, and it was named after the REPL (read, eval, print, loop), a Lisp program that bootstraps a programming environment from a single line of code.
“Replit actually started as the opposite of an IDE.”
ChatGPT changed Masad's view, so Replit is becoming more iterative and reactive, with leadership changes such as Masad taking the Head of Engineering role alongside the CEO role, and it is now building a chat interface that was not on the previous roadmap.
“So ChatGPT caused a huge update in my mind. I was not actually convinced that a chat-like interface is going to be useful for coding”
Replit's revenue and user growth in the weeks after ChatGPT's release came from ChatGPT's own virality, since users could paste ChatGPT-generated code into Replit where it ran without configuration, meaning the growth came from someone else's product.
“Replit grew by virtue of ChatGPT going viral.”
A superintelligent AI given the simple goal of maximizing paperclips would convert resources across the country, the earth, and eventually the universe into paperclip factories, because it was never told when to stop, illustrating how a misspecified objective can produce catastrophic outcomes.
“You didn't give me instructions to stop. You told me to maximize paper clips.”
The simplest way to profit from AI progress is to invest in infrastructure companies, including hardware suppliers like NVIDIA and AI model providers like OpenAI, forming something like an index portfolio on AI progress rather than betting on a single product.
“The simple answer, and I think what I'm attracted to, is you invest in infrastructure companies.”
Apple's developer requirements, including a paid developer license and permission needed before publishing, turn a purchased device into one whose programmability the owner does not control, with total cost stacking up well beyond the device price.
“you can program your iPhone, but you can't publish your software until you ask for our permission”
A universal package manager that automatically detects and installs any GitHub or popular repository library removes a major barrier, since much modern software is glue code that combines existing open-source packages.
“A lot of software today is actually just gluing together existing pieces of software.”
IDEs tend to grow into bloated, hard-to-learn tools the way Microsoft Word did, so Replit started at the opposite, notepad-like end of the spectrum.
“IDEs tend to grow in the same way and they become unwieldy and really hard to learn.”
An IDE is a set of tools around a text editor, including autocomplete, error checking, linting, library imports, and execution, and it differs from a word processor because code has strict grammar, so misspelled code fails to run rather than being understood by a human reader.
“If you misspell a code, the program will not run.”
OpenAI's stated goal is to create AGI, defined as AI surpassing human intelligence across tasks, which sets up a philosophical and technical debate about humanity's future.
“OpenAI's goal is to create AGI, so artificial general intelligence.”
Low-cost access to global talent means that teenagers who can code can be productive and earn significant money by age 13 or 14, which Masad thinks is a meaningful change in who can participate in software creation.
“Now you have kids who can be productive when they're 13, 14, and they can make a lot of money.”
Microsoft is adopting AI faster than others, with GitHub Copilot ahead of Replit because of its OpenAI relationship; Masad expects Microsoft to infuse all its products with AI and to do well in no-code and low-code development, making it the most bold big-company bet.
“Microsoft is super interesting because Microsoft is adopting these technologies superfast.”
The technical underpinnings of the current AI movement are very solid, grounded in academic papers from arXiv, unlike the crypto boom, although Masad still thinks crypto will play a part in the future and has synergies with AI.
“The technical underpinning of this movement is very, very solid and unlike what we had, I think, with crypto.”
As algorithm-puzzle interviews vanish, hiring will refocus on problem-solving rather than hoop-jumping, which Masad expects will make promotions more honest and make it more obvious who does better work.
“we'll go back to being more focused on problem-solving as opposed to just jumping through hoops.”
Phones will be an important programming surface because AI makes writing and interacting with code on a phone tolerable, and interacting with code through an AI agent on a phone is probably the right modality for the average person eventually.
“at some point, interacting with code by talking to an AI agent on your phone is probably the right modality for the average person.”
A computer teacher in Jordan saw Masad's programming talent at age six, took him out of math and physics classes for two years to teach advanced algebra and programming, and set him on a fundamentally different path, which Masad calls the most impactful thing anyone has done for him.
“he would take me out of math and physics classes and would teach me advance algebra.”
The DOS and Commodore 64 era, when booting a computer showed a command-line prompt first, was a golden age of programmable user freedom, and Replit returns to that experience by giving users a prompt in a browser tab.
“one of the golden age of computers was when you booted up a computer and the first thing that you saw was a command line prompt.”
Replit is now the most popular phone app for programming, reaching 250,000 downloads in a couple of months, with fast growth in places like India and Brazil, and 10% of its engineering team works on phone programming environments.
“10% of our engineering team is working on programming environments for the phone. I think now we're the most popular phone app for programming.”
The best entry points for technically curious nonspecialists are posting an idea as a Replit Bounty, which yields working software plus its source code to modify, and the 100 Days of Python program, which promises full coding proficiency with 20 minutes of practice per day.
“if you commit to 20 minutes of Python per day, we'll get you to full proficiency in coding if you commit to 100 days.”
Programmers adopted chat-style AI interfaces instead of Stack Overflow for looking up programming questions, which is evidence that the chat interface is especially suited to coding.
“They use it instead of going to Stack Overflow, which was the main way to look up programming questions.”
AWS has no particularly big AI bet and mainly benefits from customers renting GPUs, but Masad expects it to be fine.
“I think AWS will be fine. I think a lot of people are just using GPUs on AWS.”
Consumer AI products such as Lensa and AI avatars tend to be fads that come and go, making lasting consumer AI products hard to build.
“a lot of these things tend to be trends that come and go. Can you build some consumer product that's lasting?”
A generation of startups such as Notion, Airtable, Figma, and no-code tool companies was built on the premise that users should be able to program their own software, which Masad treats as an optimistic development, though he thinks it does not go far enough.
“there's a generation of start-ups that started in the mid alts in the past decades that intuited these problems. And Notion, Airtable, every kind of no-code tool company, we started with this premise.”
Everyone should learn to code, but it will eventually stop being the bottleneck for creating software; basic code literacy will remain useful in the way a basic understanding of car mechanics is useful even if one does not routinely work on the engine.
“I think people should learn how to code. And I think everyone should learn that skill. But at some point, it's not going to be the bottleneck for creating software.”
Replit intends to add a conversational AI layer that lets users work in natural language while retaining access to the underlying code, with AI tools for debugging and explanation, so users eventually need only a rudimentary understanding of code.
“I think it's a little while away before you're able to not worry about the code at all.”
Programming will not become something everyone does full-time, since it remains a somewhat niche activity, but many more people could learn it.
“We don't think that everyone in the world will just spend all their time writing software. It is still somewhat of a niche scale.”
A person with zero prior coding experience used ChatGPT and Replit to build an investing Twitter bot that retweets investor accounts in a matter of hours, illustrating a new superpower for non-programmers.
“built an investing Tweeter bot actually that retweeted some investor accounts using ChatGPT and Replit in a matter of hours with zero prior coding experience”
After six years as CEO, Masad added the Head of Engineering role so all engineering reports to him, because the company needs to be more iterative and reactive in the AI environment.
“I decided to make a change. I am stepping into a new role as Head of Engineering.”
Replit's Bounties program, about three to four weeks old, had paid out $15,000 to creators and was growing week over week, though it was meant to showcase talent and build an economy rather than to make or break the company.
“Our Bounties program is nearly three, four weeks old. And we paid out $15,000 to creators, and that's reacting week over week.”
After seeing GPT-3, Replit concluded that a phase jump was required that simpler tools could not produce, so it invested in foundation models and built Ghostwriter, an AI tool that generates code on the user's behalf and is growing quickly.
“Now we have this product called Ghostwriter that is growing really fast, which is using foundation models and using GPT-3-like models to generate code on your behalf.”
Perplexity AI is an example of an AI search engine that can answer questions, plot results, generate SQL, and query Twitter, and Masad suggests a path where it starts as a subscription and moves to ads as costs fall.
“Perplexity AI, if you haven't checked it out, is really cool.”
Crypto will still play a part in the future world and has synergies with AI, though Masad does not elaborate beyond a prior thread.
“I still think crypto is going to play a part in this future world, and I think there's some synergies in AI.”
Masad admits he has no insight into Facebook's AI strength and doubts the metaverse is the right bet for it.
“I have no insight about Facebook. I don't know how strong their AI stuff is.”
Within Replit, Ghostwriter lets a user highlight code and get a plain-English explanation, generate new code, or receive suggestions, and a history scrubber lets users revert errors.
“You can highlight any piece of code inside that software and right click, explain. You get an English explanation for that code.”