kernel fusion
Combining multiple computation steps into a single GPU kernel to avoid redundant memory reads/writes, enabled in tinygrad by lazy evaluation.
As defined in George Hotz #3
Where it comes up · 2
Most important claims · 3 of 3Search all claims for “kernel fusion” →
- Tinygrad uses laziness like Haskell: instead of computing A times B immediately and storing it to memory, it waits until the user realizes the tensor, allowing the plus C operation to be fused into one kernel, which eliminates the excessive loads and stores to memory that PyTorch incurs.George Hotz #3
- Engineers should no longer hand-write GPU kernels; instead they design the strategy on a whiteboard, describe the input, output, and work dispatch in natural language, and a model executes the implementation.Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]
- Kernel work is not finished because new model features like changed positional embeddings (RoPE variants) break existing fused GPU kernels, forcing patches, and the ability to quickly modify kernels is valuable even though new ones are rarely invented from scratch.Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]