factual

Removing facts from models is nearly impossible

Censoring or removing specific facts (like Tiananmen Square) from a language model is practically impossible because you'd have to filter them out of the pre-training data at every stage, and quality filters and word-play workarounds make complete removal infeasible; the internet's data also imparts a slight inherent left bias regardless of intent.

factualpending

Speaker

Nathan Lambert

Evidence Quote

“you have to do it at every stage, you have to do it at the pre-training... most people think that pre-training is where most of the knowledge is put into the model”

Source

Deepseek Dylan Patel Nathan Lambert— Lex Fridman Podcast
Created: 6/13/2026, 3:31:25 AM

Your notes

Loading notes...