factual
SRAM wafer-scale speed comes with a KV cache ceiling
Wafer-scale SRAM architectures can host a trillion-parameter model's weights and reach about 1,000 tokens per second because weights stream in about a millisecond, but the growing, dynamic KV cache is a persistent limit those chips handle poorly.
factualpending
Speaker
Neil MovvaEvidence Quote
“you can't do much about the KV cache very easily. The KV cache is something that grows as people use the model more”
Created: 11/17/2026, 6:43:49 AM
Your notes
Loading notes...