factual
OCR Generates Trillions of Tokens of Synthetic Data
Labs build OCR systems (DeepSeek OCR, OLMo OCR) because vast amounts of PDFs and digital documents on the web aren't easily encoded with text; extracting them yields what can be trillions of tokens of candidate data, which is described as synthetic data for pre-training and is a small percentage of the potential funnel that actually gets used.
factualpending
Speaker
Nathan LambertEvidence Quote
“the data you actually train the model on is a small percentage of this.”
Created: 6/13/2026, 3:24:41 AM
Your notes
Loading notes...