New Quantization Techniques Released for Qwen3.8-Flash-Next

The ISTA Deep Algorithms and Systems Lab has announced the release of new quantized versions of the Qwen3.8-Flash-Next model, utilizing advanced Gumbel-Softmax Quantization (GSQ) and Riemannian Constrained Optimization (RCO). The release includes several GGUF builds and an experimental “expert-pruned” version specifically targeted at coding tasks.

The base Qwen3.8-Flash-Next is a sparse mixture-of-experts (MoE) model featuring 48 layers and 512 routed experts per layer. The full-precision BF16 version contains 176.9 billion parameters and requires 354 GB of memory.

Technical Advancements in Quantization

The new release introduces GSQ, a post-training scalar quantization method that learns grid assignments and group scales jointly. According to the developers, this approach aims to minimize the performance gap between scalar and vector quantization at low bit-widths while remaining compatible with standard GGUF deployment formats.

The RCO method is utilized to enforce exact parameter budgets via gradient descent on task loss. In this release, RCO serves two functions: assigning specific quantization types to every tensor and selecting which experts to retain in the pruned Coder build by optimizing KL divergence against the unpruned model.

Performance benchmarks provided by the developers indicate that the 3.50 bpw (bits per parameter) IQ3_S build matches the BF16 base model across several metrics. Specifically, it achieved a score of 92.93 on GPQA-Diamond, compared to 91.92 for the BF16 version, and a 100.00 score on AIME25. At a significantly reduced size of 2.40 bpw (66.4 GB), the model reportedly maintains a zero-shot average of 78.00, exceeding the BF16 value of 76.94.

Experimental Expert-Pruned Coder Build

In addition to standard quantization, the lab released an experimental Coder build that employs expert pruning. Rather than reducing the precision of all parameters, this version removes 256 of the 512 experts per layer.

The resulting model has a total size of 58.4 GB, with a resident working set of 29.6 GB required for operation. This reduced footprint is designed to fit within the capacity of a single 32 GB accelerator. The developers note that the combined effect of pruning and quantization results in an average of 1.89 bits per parameter relative to the original transformer.

Benchmark results for the Coder build include:

  • SWE-bench Verified: 75.60 (retaining 91.3% of the BF16 performance of 82.80).
  • LiveCodeBench v6: 86.28 (retaining 98.7% of the BF16 performance of 87.43).

Qwen Architecture in Audio Modeling

Separate data mapping from the audio.cpp project suggests that the Qwen model family is increasingly utilized as a linguistic backbone for audio-based artificial intelligence. According to a report shared on Reddit, 32 audio model families utilize a Qwen-family architecture, with 20 of those specifically employing the Qwen3 LLM.

The architecture is reportedly being applied across various modalities, including speech synthesis, automatic speech recognition (ASR), music generation, and audio/video models. The mapping indicates that Qwen-based architectures are becoming a common component in the development of over 100 different audio models.