factual

OCR Generates Trillions of Tokens of Synthetic Data

Labs build OCR systems (DeepSeek OCR, OLMo OCR) because vast amounts of PDFs and digital documents on the web aren't easily encoded with text; extracting them yields what can be trillions of tokens of candidate data, which is described as synthetic data for pre-training and is a small percentage of the potential funnel that actually gets used.

factualpending

Speaker

Nathan Lambert

Evidence Quote

“the data you actually train the model on is a small percentage of this.”

Source

Ai Sota #2026— Lex Fridman Podcast
Created: 6/13/2026, 3:24:41 AM

Your notes

Loading notes...