A dual Nvidia RTX 6000 workstation can generate 324 tokens per second with a 1 billion parameter model, sustaining over 300 tokens per second consistently.

factualpending

Speaker

Dave

Evidence Quote

“345 tokens per second... it's able to easily sustain over 300 tokens per second”

Source

Feed Your OWN Documents to a Local Large Language Model!— Dave's Garage
Created: 8/11/2026, 5:30:47 AM

Your notes

Loading notes...