Thread Ripper
A high-end AMD CPU line designed for workstations and professional computing, featuring high core counts (typically 24-96 cores) and substantial cache for multi-threaded performance
As defined in Run Local LLMs on Hardware from $50 to $50,000 - We Test and Compare!
Other definitions
A high-end CPU line from AMD designed for multi-threaded computing tasks, suitable for content creation, virtualization, and workstation applications
Feed Your OWN Documents to a Local Large Language Model!
Where it comes up · 2
- Run Local LLMs on Hardware from $50 to $50,000 - We Test and Compare!6 claims
- Feed Your OWN Documents to a Local Large Language Model!1 claim
Most important claims · 7 of 7Search all claims for “Thread Ripper” →
- A Threadripper processor paired with dual RTX 6000 GPUs uses almost no CPU utilization during large language model generation, suggesting that GPU acceleration is the dominant factor in LLM inference performance and CPU is not a bottleneck.Feed Your OWN Documents to a Local Large Language Model!
- The Llama 3.1 405 billion parameter model requires 228GB of download storage and approximately 200GB of RAM to load, but produces only one token every several seconds on a $50,000 Dell workstation with 512GB RAM and dual overclocked Thread Ripper 96-core CPU with RTX 6000 Ada GPU, taking approximately 30 minutes to generate a simple response.Run Local LLMs on Hardware from $50 to $50,000 - We Test and Compare!
- The episode received viewer criticism in comments for two specific choices: testing primarily on expensive hardware (Dell Thread Ripper with dual RTX 608 series GPUs) and using a complex setup (Linux on WSL2 on Windows).Run Local LLMs on Hardware from $50 to $50,000 - We Test and Compare!
- A 4-year-old Thread Ripper 3970x CPU with 32 cores and 128GB RAM paired with an Nvidia RTX 4080 GPU serves as a reasonable functional equivalent to a contemporary gaming PC despite its age.Run Local LLMs on Hardware from $50 to $50,000 - We Test and Compare!
- The Thread Ripper workstation with 512GB of RAM has sufficient memory to load the full 405 billion parameter Llama 3.1 model, with peak memory usage reaching close to 200GB.Run Local LLMs on Hardware from $50 to $50,000 - We Test and Compare!
- The high-end workstation configuration tested features an overclocked 96-core Thread Ripper CPU set to run at more than 800 watts TDP, consuming more than 1,200 watts total at the wall when combined with the Nvidia RTX 6000 Ada GPU.Run Local LLMs on Hardware from $50 to $50,000 - We Test and Compare!
- Dave purchased his Thread Ripper 3970x workstation approximately four years ago, and while it retains a single-core speed limitation from that era, it remains capable when paired with modern GPUs.Run Local LLMs on Hardware from $50 to $50,000 - We Test and Compare!