Computer use
An AI feature allowing a model to operate a computer interface directly (clicking, typing, navigating) to perform tasks like booking flights.
As defined in The Past and Future of AI (with Dwarkesh Patel)
Other definitions
A capability allowing Claude to control mouse, keyboard, and screen to interact with any application on a computer by visually perceiving the screen and clicking/typing to navigate and complete tasks, serving as a fallback when API-based tools are unavailable.
How to Use Claude's Massive New Upgrades
Where it comes up · 6
- How to Use Claude's Massive New Upgrades6 claims
- Ep19. State of Venture, AI Scaling, Elections | BG2 w/ Bill Gurley, Brad Gerstner, & Jamin Ball3 claims
- Dario Amodei2 claims
- Andrej Karpathy — AGI is still a decade away1 claim
- Dwarkesh Patel Podcast - Dario Amodei — "We are near the end of the exponential"1 claim
- The Past and Future of AI (with Dwarkesh Patel)1 claim
Most important claims · 10 of 14Search all claims for “Computer use” →
- Computer use capability allows Claude to control mouse, keyboard, and screen to interact with any application by clicking, navigating, and typing, enabling task completion in non-API-enabled legacy software without requiring setup or pre-built connectors.How to Use Claude's Massive New Upgrades
- A 'country of geniuses in a data center' will be able to do context-dependent jobs like video editing by having general control of a computer screen—browsing the web, reviewing past interviews and audience feedback, querying you and your staff, and reviewing edit history—but this depends on computer use reaching a point of reliability, which is currently one of the things blocking deployment (OSWorld benchmark climbed from ~15% to 65-70% in about a year).Dwarkesh Patel Podcast - Dario Amodei — "We are near the end of the exponential"
- Computer use announcement tweet reached 40 million views and 62,000 bookmarks within 16 hours of posting, indicating exceptional viral engagement and high market interest in the capability.How to Use Claude's Massive New Upgrades
- Current agents like Claude and Codex are impressive but still don't work as reliable employees because they lack intelligence, multimodality, computer use, and continual learning; working through all these deficits will take roughly a decade, not a year.Andrej Karpathy — AGI is still a decade away
- Computer use and code writing capabilities represent ultimate primitives for agent systems because most knowledge work requires hopping between multiple applications and handling diverse data sets, and agents need system traversal ability to automate real enterprise work beyond single-system optimization.How to Use Claude's Massive New Upgrades
- Anthropic released a 'computer use' capability allowing Claude to take control of a user's browser and complete tasks like booking hotels or airline tickets, and Google reportedly was expected to release a similar capability called 'Jarvis' this week, indicating that agentic AI (AI that can autonomously perform multi-step tasks) is the next frontier of AI capability deployment.Ep19. State of Venture, AI Scaling, Elections | BG2 w/ Bill Gurley, Brad Gerstner, & Jamin Ball
- Humanoid robots like Tesla's Optimus are preferred over non-humanoid robotic forms because humanoid robots work in environments (the physical world) that were already designed for human bodies, requiring no infrastructure redesign, whereas agentic AI using 'computer use' capability succeeds because it operates on existing web infrastructure designed for human interaction, providing similar 'use-as-is' advantage without requiring system redesign.Ep19. State of Venture, AI Scaling, Elections | BG2 w/ Bill Gurley, Brad Gerstner, & Jamin Ball
- Computer use is particularly valuable for enterprises that depend on legacy software not designed for API integration, where traditional automation approaches cannot reach, making it a competitive advantage for organizations without modern tech stacks.How to Use Claude's Massive New Upgrades
- A new browser company called Browserbase and conceptually similar approaches are exploring whether to build a new browser designed for agents (with composable APIs instead of buttons) rather than humans, representing an alternative architecture to the 'computer use' approach of scraping human-designed websites and clicking buttons, but this would require rebuilding all underlying transaction infrastructure (like Stripe) to work with agents.Ep19. State of Venture, AI Scaling, Elections | BG2 w/ Bill Gurley, Brad Gerstner, & Jamin Ball
- Computer use required little additional training because a strong pre-trained model is 'halfway to anywhere' in intelligence space (like reaching low-earth orbit being halfway to anywhere); generalization means screenshots-as-images plus click/keyboard outputs in a loop produces full interactive computer control.Dario Amodei