NVIDIA Releases Nemotron-Labs-3-Puzzle-75B-A9B: A Compressed Hybrid MoE LLM Delivering 2.03x Server Throughput at Matched User Throughput

Nvidia has released Nemotron-Labs-3-Puzzle-75B-A9B, a compressed version of its large language model that uses structural optimization to reduce total parameters from 120.7 billion to 75.3 billion. By utilizing this hybrid architecture, the model achieves a two-fold increase in server throughput while maintaining performance levels comparable to the original version.