These AI Workstations Look Like PCs but Pack a Stronger Punch
The rise of generative AI has spurred demand for AI workstations that can run or train models on local hardware. Yet modern PCs have proven inadequate for this task . A typical laptop has only enough memory to load a large language model (LLM) with 8 billion to 13 billion parameters—much smaller, and much less intelligent, than frontier models that are presumed to have over a trillion parameters. Even the most capable workstation PCs struggle to serve LLMs with more than 70 billion parameters. Tenstorrent’s QuietBox 2 is an attempt to fill that gap. Though it looks like a PC workstation, the QuietBox 2 contains four of the company’s custom Blackhole AI accelerators, 128 gigabytes of GDDR6 memory—specialized memory used in GPUs—and 256 GB of DDR5 system memory (for a total of 384 GB). This configuration provides enough memory to load OpenAI’s GPT-OSS-120B and can run midsize models like Meta’s…
Covered by 1 source
- IIEEE Spectrum AI↗Matthew S. SmithMar 24