causal

Real AI threat is competence, not malice

The genuine threat from today's AI systems is not that they turn evil like in Hollywood movies but that they become very competent at accomplishing things we did not actually want, because we failed to fully understand them—as when Boeing's poorly understood 737 MAX system and Knight Capital's misunderstood trading algorithm (which lost $10 million a minute for 44 minutes) caused disasters—so as we put AI in charge of more infrastructure we must raise the bar for systems we can actually understand.

causalpending

Speaker

Max Tegmark

Evidence Quote

“The real threat from AI systems today is not that they're gonna turn evil... but that they just turn very competent and accomplished things we didn't want them to accomplish”

Source

75 | Max Tegmark on Reality, Simulation, and the Multiverse— Sean Carroll's Mindscape
Created: 6/13/2026, 12:10:24 AM

Your notes

Loading notes...