causal
Cache non-determinism vs scratchpad determinism
The biggest source of non-deterministic latency in a CPU is the cache, which transparently decides in hardware whether data comes from fast cache or slow DDR — a decision that depends on ambient conditions like other running programs and the cache's random replacement, making runtime unpredictable; TPUs instead use a scratchpad model where separate explicit instructions target scratchpad versus HBM, baking the decision into software for deterministic latency.
causalpending
Speaker
Reiner PopeEvidence Quote
“whether or not you get a cache hit depends on the ambient environment of the CPU: what other programs are running, what has run recently, and what the random number generator inside the cache system is doing. That is a big source of non-determinism in the runtime of a CPU.”
Created: 6/13/2026, 3:33:43 AM
Your notes
Loading notes...