factual
Removing facts from models is nearly impossible
Censoring or removing specific facts (like Tiananmen Square) from a language model is practically impossible because you'd have to filter them out of the pre-training data at every stage, and quality filters and word-play workarounds make complete removal infeasible; the internet's data also imparts a slight inherent left bias regardless of intent.
factualpending
Speaker
Nathan LambertEvidence Quote
“you have to do it at every stage, you have to do it at the pre-training... most people think that pre-training is where most of the knowledge is put into the model”
Created: 6/13/2026, 3:31:25 AM
Your notes
Loading notes...