Transformers now runs llama.cpp quants
The Hugging Face Transformers library has added native support for GGUF-formatted quantized models from llama.cpp. This integration allows developers to load compressed weights directly through the Transformers API, simplifying the process of running memory-efficient models on consumer hardware.
Covered by 1 source
- HHugging Face Blog↗Sep 22