A dual Nvidia RTX 6000 system can generate over 300 tokens per second when running a 1 billion parameter model like Llama 3.2, sustaining approximately 324 tokens per second on longer text generation tasks.
factualpending
Speaker
DaveEvidence Quote
“345 tokens per second... 324 so it's able to easily sustain over 300 tokens per second”
Created: 8/11/2026, 7:47:33 AM
Your notes
Loading notes...