Developments in Qwen Model Architectures and Conversions
Recent activity within the open-source AI community has highlighted significant technical shifts in the Qwen model ecosystem, ranging from experimental Mixture-of-Experts (MoE) conversions to unverified indications of upcoming model generations.
Experimental MoE Conversions for Qwen3
A developer identified as bayliner1980 has released a new conversion of the Qwen3-0.6B model into a Mixture-of-Experts (MoE) architecture. According to reports on Reddit, this conversion utilizes a “pseudo-shared expert” to mitigate quality loss caused by the absence of a native shared expert in the Qwen3 architecture.
Benchmark data provided by the developer indicates that the MoE version performs slightly lower than the dense version on Wiki ppl (26.98 vs 23.88) and C4 ppl (36.32 vs 31.63). In terms of throughput, the MoE model achieved 352 tokens per second (t/s) on GPU and 124 t/s on CPU, compared to 377 t/s and 115 t/s for the dense model, respectively. The developer noted that at this low parameter count, the conversion does not yield significant speed improvements and cautioned that the model may struggle with factual accuracy and coding tasks.
Performance Benchmarks of Qwen 3.8 FN Engines
Comparative testing of various engines running Qwen 3.8 FN models on an NVIDIA RTX 5090 has provided data on inference efficiency. According to a benchmark report by the developer of the Basalt engine, Basalt and Infernix emerged as the highest-performing engines in terms of decode speed.
In short-context decode tests, Basalt achieved 354 t/s, while Infernix reached 349 t/s. As context depth increased to 200K tokens, Basalt maintained a decode speed of 160 t/s, whereas llama.cpp—running without Multi-Token Prediction (MTP)—recorded 32 t/s. Prefill performance also showed significant variance; Basalt reached 8,609 t/s at 200K context, while llama.cpp recorded 1,257 t/s.
Other engines tested included capy-flash, Strata, ninfer-ext, and ninfer. The results suggest that specialized engines utilizing MTP and optimized conversions for NVIDIA’s NVFP4 format offer substantial advantages in both prefill and decode speeds over standard implementations.
Unverified Reports of Qwen4 Development
Speculation regarding the next generation of models has surfaced via recent GitHub activity. A pull request in the vllm-xpu-kernels repository suggests the existence of a Qwen4-27B model, with observations noting that the KV cache size for this version does not appear to be decreasing.
Additionally, new GitHub accounts have been observed contributing to repositories related to the t-head/pla project. While some community members suggest these may be indicators of an upcoming Qwen4 release, the age of these accounts and the minimal nature of the commits mean these developments remain unverified and lack independent confirmation from Alibaba.