Discrepancies Emerge Over Bonsai 2 Model Performance Claims
PrismML has released Ternary Bonsai 2 27B, a heavily quantized version of the Qwen 3.8 27B model. While the developer claims the model retains 98.2% of “intelligent” performance compared to its full-precision counterpart, recent scrutiny from the developer community suggests these figures may be based on selective data.
Benchmark Discrepancies and Data Accuracy
According to discussions on Reddit and analysis of the model’s GitHub whitepaper, critics have pointed to a significant gap between the marketing claims and the technical documentation. While the 98.2% figure is highlighted in promotional materials, the whitepaper itself notes that on specific benchmarks, the model retains a lower percentage of performance.
On the Terminal-Bench 2.1 and SWE-bench Verified benchmarks, Ternary Bonsai 2 27B achieved scores of 52.8 and 60.8, respectively. In comparison, the full-precision Qwen 3.5-27B (noted by some users as a typo for Qwen 3.8) scored 69.7 and 80.6. This indicates the Bonsai model retains approximately three-quarters of the full-precision performance on these specific metrics.
Furthermore, some users have noted that the communication regarding the model’s capabilities lacks transparency, suggesting that the actual performance degradation is more pronounced than the headline figures suggest.
Comparative Performance in Local Environments
Despite the controversy regarding benchmark accuracy, independent testing has highlighted the model’s utility for local hardware. In a comparative test involving an RTX 5090 GPU, a user reported that Ternary Bonsai 2 27B demonstrated high levels of autonomy and detail when tasked with generating complex code for a 3D scene.
In that specific test, the Bonsai model produced 106,396 output tokens over approximately 17.5 minutes, achieving a speed of 101 tokens per second. While the output was described as more detailed than competitors like Gemma 4 12B and Qwen 3.5 9B, the test also highlighted a significant trade-off in efficiency. The Bonsai model took significantly longer to complete its task compared to the Qwen 3.5 9B model, which completed its run in 70 seconds.
Implications for Local AI Deployment
The release of Bonsai 2 highlights an ongoing tension in the local AI community between model intelligence and computational efficiency. The ability to run a 27B parameter model with high reasoning capabilities on consumer-grade hardware, such as an RTX 3060, remains a primary interest for enthusiasts.
However, critics argue that the “brittle” argument for the model’s necessity is weakened by the existence of existing inference engines designed to run larger Mixture-of-Experts (MoE) models on mobile devices and smartphones. As the industry moves toward more efficient quantization, the debate continues over whether highly compressed models like Bonsai 2 offer genuine intelligence or merely a highly optimized approximation of existing frontier models.