vllm v0.32.0
# v0.32.0 ## Highlights This release features 814 commits from 363 contributors (121 new)! * **Structured decisions and generic parsers**: the new endpoint answers , and questions over a structured state (#59299), is served by default (#60677, #60696), and also runs on (#61064), (#60676) and DiffusionGemma (#60466). The new tool and reasoning parser (, ) reads the checkpoint's , so models that ship it need no model-specific parser (#58604, #58602), in both the Python and Rust frontends (#59005). * **Low-precision KV cache everywhere**: NVFP4 KV cache now runs on SM8x and SM12x (#46963), SM90 (#60620) and with XQA decode on SM12x (#60452), and on MiniMax-M3 sparse attention on SM100 (#59300); GLM-5.x defaults to an FP8 KV cache on SM100 for 2.3-5.5% more throughput (#60140, #60281); on SM90 gives GLM-5.3-Flash up to 31% more output throughput (#59246); ROCm gains the 4-bit KV cache with about 2x the density of FP8 (#57057).…
Covered by 1 source
- GGitHub Releases↗1d ago