OpenAI continuously innovates beyond basic scaling, including data formatting optimizations, architectural shifts like LSTM to transformer, and continued evolution beyond the 2017 transformer paper—these innovations amplify the base scaling law effect.
causalpending
Speaker
Greg BrockmanEvidence Quote
“we absolutely have new ideas that are constantly powering what we do... We have made tons of innovations and we constantly are improving things. And sometimes these are micro tweaks... Sometimes it's larger. You think about the shift from the LSTM to the transformer...”
Created: 8/12/2026, 6:18:53 PM
Your notes
Loading notes...