New performance benchmarks for the Qwen-Flash-Next model have emerged from various independent testing environments, highlighting significant variations in speed and efficiency depending on hardware configurations and software optimization techniques.

High-Performance Benchmarks on Next-Generation Hardware

According to reports shared on Reddit, the Qwen3.8-Flash-Next model demonstrated extreme performance capabilities when paired with specialized software and high-end hardware. Using the Basalt engine on an NVIDIA RTX 5080 (16 GB VRAM) and a Core Ultra 9 285K processor, testers reported prompt processing (PP) speeds reaching an average of 5,475 tokens per second (t/s) and peaks of 5,662 t/s.

In these specific tests, which utilized an IQ3_S quantization and a 262K context window, generation speeds (TG) averaged approximately 154.9 t/s, with peaks reaching 262 t/s. These results were observed during complex workloads, including agent coding sessions and context sweeps.

Optimization via Selective Offloading and Quantization

Other independent developers have demonstrated that the model can be run on more modest hardware through advanced memory management. One user reported achieving 70 t/s for prompt processing and 8 t/s for token generation using an 8 GB VRAM and 24 GB RAM setup. This was achieved via llama.cpp by employing a specific methodology: keeping the smallest Mixture-of-Experts (MoE) in VRAM for speed and storing the n-gram data on a high-speed SSD.

The model used in these tests was a “Coder” variant by ISTA DASLab, utilizing GSQ RCO quantization at 3.5 bits per weight (BPW). While the model’s effective rate is reported as 1.89 bits per parameter due to the removal of half of its experts, the retained weights remain at 3.5 BPW.

Low-Memory and SSD-Based Implementations

A third approach, documented on GitHub by developer MaxDevv, focuses on “Lean-QwenFlash,” a method designed to run the model on machines with limited system RAM by utilizing direct NVMe SSD offloading.

In tests conducted on an NVIDIA RTX 4090 with only 8 GB of system RAM allocated, the implementation achieved:

  • Short context (8k): 28–33 t/s for prose and 19–22 t/s for code.
  • Full context (262k): 20.0 t/s for decoding and approximately 1,627 t/s for prefill.

On a laptop equipped with an RTX 3080 (16 GB VRAM) and 32 GB of RAM, the model maintained approximately 18–21 t/s for decoding at a 100k context window.

Technical Considerations and Limitations

The reported benchmarks highlight several technical dependencies. Performance is heavily influenced by the choice of quantization, the use of specific chat templates, and the speed of the storage medium. Users noted that the first run of the model may be slower due to memory-mapped (MMAP) file loading.

While these results suggest significant potential for running large-scale models on consumer-grade hardware, it should be noted that these benchmarks are derived from self-published reports on platforms like Reddit and GitHub and have not undergone independent, third-party verification.