Local Hardware Optimization for Qwen 3.8 Flash Next

A user on Reddit has reported significant performance benchmarks for the Qwen 3.8 Flash Next model when running on consumer-grade hardware using the Strata software framework. According to the unverified report, a system equipped with 8 GB of VRAM and 32 GB of RAM achieved prompt-processing (PP) speeds of up to 900 tokens per second (t/s) and token-generation (TG) speeds of 30 t/s.

The tests were conducted on a Linux-based system featuring an AMD Ryzen 9 8945HX CPU and an RTX 5060 Laptop GPU. The user utilized the “Gyro S” quantization of the model, claiming that this specific version provided a superior balance of quality and speed compared to other available quants such as GSQ-RCO Coder or GSQ-RCO IQ2_XS.

Technical Configuration and Performance Metrics

The reported performance gains appear to stem from specific memory management configurations within the Strata framework. Key parameters used to optimize the workload across VRAM, RAM, and SSD storage included:

  • Memory Distribution: The user utilized --mmap-experts to place non-resident experts on the SSD and --kv-resident 20480 to distribute the KV cache between VRAM and RAM.
  • VRAM Management: A reserve of 500 MB to 900 MB was maintained for system tasks, and the --pcie-frac parameter was tuned to optimize data transfer over the PCIe bus.
  • Context Handling: To manage large context windows without exhausting memory, the user employed --conversation-cache rather than parallel processing, which can trigger out-of-memory (OOM) errors at context sizes exceeding 131K tokens.

Under the “Chat” configuration with a 131K token context, the user reported PP speeds between 930 and 1,040 t/s and TG speeds of 30–45 t/s. In an “Agentic” configuration, the reported PP speeds were lower, ranging from 670 to 890 t/s, while TG remained between 30 and 45 t/s.

Comparative Results and Real-World Application

The report claims that these results outperform previous setups using the Qwen 3.6 35B A3B model. In comparison, the user noted that their Qwen 3.6 35B A3B setup achieved PP speeds of 630–680 t/s and TG speeds of 27–35 t/s.

In practical testing involving coding tasks and agentic workflows, the user reported that the Qwen 3.8 Flash Next model improved development speed and code quality. However, it should be noted that these results are based on individual testing and have not been independently verified by third-party benchmarks.

Broader Context of Chinese AI Development

The performance of the Qwen series, developed by Alibaba, occurs amid a broader expansion of the Chinese AI sector. Recent industry data indicates that Chinese AI models have reached significant scale, with reports suggesting they account for 4.69 trillion weekly tokens. This development follows various technological advancements in the region, including the launch of open-source models and the development of specialized hardware to navigate international chip restrictions.