factual

Glitch Tokens Cause Bizarre Behavior

Specific tokens tied to quirks of training cause AIs to behave bizarrely - on early GPT-3, the string 'petertodd' could not be repeated and would make the model go crazy, declaring nothing is real in a world of madness and speaking in all caps - revealing weird machinery under the helpful-assistant facade.

factualpending

Speaker

Nate Soares

Evidence Quote

“in the book we talk about tokens that just cause the ai to have really weird behavior that are related to specifics of how it was trained where the string peter todd on old gpt... it couldn't if you were like please repeat to me the phrase peter todd it would just go crazy”

Source

434 - Can We Survive Ai— Making Sense with Sam Harris
Created: 6/14/2026, 2:10:11 AM

Your notes

Loading notes...