Llama.cpp Pull Request Introduces Support for Maple 20B-A1B Model

Technical Integration of Maple 20B-A1B Architecture

A new pull request in the llama.cpp repository, submitted by contributor AlexGabbia, marks a significant technical development for the open-source inference engine. The pull request (#27000) introduces support for the Maple 20B-A1B architecture, a ternary Mixture-of-Experts (MoE) model. According to documentation on GitHub and discussions on Reddit, the current implementation focuses on CPU support for the model.

The Maple 20B-A1B model is being developed by DeepGrove, and its integration into the ggml-org/llama.cpp ecosystem is intended to facilitate efficient local execution. While the current pull request addresses CPU capabilities, developers are reportedly working on CUDA and Vulkan backends to enable hardware acceleration via NVIDIA and other graphics processing units.

Performance Benchmarks and Hardware Compatibility

Preliminary performance data shared by DeepGrove on Hugging Face suggests high throughput for the model on specific hardware configurations. Unverified benchmarks indicate the model can achieve speeds exceeding 200 tokens per second (t/s) on a Mac Mini M4 and over 120 t/s on an iPhone.

The model is being released in GGUF format, a standardized file format used by llama.cpp to enable efficient quantization and deployment. This format is particularly relevant for users with limited Video RAM (VRAM), as the MoE architecture is designed to optimize resource allocation. Industry observers on social media platforms have noted that the model’s efficiency may make it a competitor to other lightweight models, such as Ling 3 Tiny, though specific comparisons regarding KV cache memory consumption remain a subject of ongoing discussion among users.

Implications for Local LLM Deployment

The introduction of ternary MoE architectures into mainstream inference tools like llama.cpp highlights a continuing trend toward optimizing large language models for consumer-grade hardware. By utilizing ternary weights and MoE structures, the Maple 20B-A1B aims to balance model capacity with computational efficiency.

While the performance figures cited by the developers have not been independently verified by third-party testing labs, the integration into the llama.cpp codebase provides a framework for broader community testing. As CUDA and Vulkan support is finalized, the model’s ability to run on a wider array of GPU-accelerated devices will likely be determined.