A dual Nvidia RTX 6000 system can generate over 300 tokens per second when running a 1 billion parameter model like Llama 3.2, sustaining approximately 324 tokens per second on longer text generation tasks.

factualpending

Speaker

Dave

Evidence Quote

“345 tokens per second... 324 so it's able to easily sustain over 300 tokens per second”

Source

Feed Your OWN Documents to a Local Large Language Model!— Dave's Garage
Created: 8/11/2026, 7:47:33 AM

Your notes

Loading notes...