New Quantization Method Released by Huawei Research

Huawei’s Computer Science and Technology Research Center (CSL) has released MACaQ, an open-source quantization method designed to reduce the memory footprint of large language models (LLMs) while maintaining performance. The project, hosted on GitHub, includes a fork of llama.cpp to support the new format.

The method aims to optimize GGUF-compatible models across various bit-widths, including 2-bit (IQ2_XXS, IQ2_XS), 3-bit (IQ3_XXS, IQ3_S), and 4-bit (IQ4_NL, IQ4_XS) formats. According to technical documentation and benchmarks shared on Reddit, the method demonstrates significant efficiency in reducing Kullback–Leibler (KL) divergence—a metric used to measure the difference between probability distributions—compared to existing quantization methods such as Unsloth and Bartowski.

Performance Benchmarks

Unverified benchmark data shared by users on Reddit suggests that MACaQ achieves higher fidelity than current industry standards at similar file sizes. In tests conducted on the Qwen3-4B model (IQ4_XS), MACaQ reported a mean KL divergence of 0.02804, compared to 0.03663 for Unsloth and 0.03443 for Bartowski, despite having nearly identical file sizes.

The performance gap appears more pronounced in larger models. For the Qwen3.8-27B model, MACaQ recorded a KL divergence of 0.036784, which is approximately 65% lower than the 0.104170 reported for Bartowski. In the case of the Qwen3.6-35B-A3B model, MACaQ achieved a KL divergence of 0.018696, compared to 0.020954 for Unsloth (UD-IQ4_XS).

However, some community members have noted that KL divergence is a “loose measure” and that the relevance of a quantization technique may depend heavily on the quality and relevance of the calibration dataset used during the process.

Technical Implementation and Scalability

The research team reports that the method is capable of handling large-scale models efficiently. Specifically, they claim to have quantized a 122-billion-parameter model in 64 minutes using 32.5 GB of GPU memory.

Users have noted that the implementation requires specific technical steps for deployment. Because the developers do not currently provide ready-made GGUF files for all models, users must compile llama.cpp with specific patches to ensure compatibility with the MACaQ format. While some files have been uploaded to Hugging Face by community members, full support for all model versions remains in development.