Release of Quantized DeepSeek-V4.1-Flash Models

The ISTA Deep Algorithms and Systems Lab has announced the release of quantized GGUF versions of the DeepSeek-V4.1-Flash-1017 model. Utilizing two specific techniques—Gumbel-Softmax Quantization (GSQ) and Riemannian Constrained Optimization (RCO)—the release aims to significantly reduce the memory footprint of the large-scale model while attempting to maintain high levels of performance.

The DeepSeek-V4.1-Flash-1017 is a sparse mixture-of-experts (MoE) model featuring 763.2 billion parameters. In its standard configuration, it utilizes 384 routed experts per layer, with six active experts per token. The released quantized versions operate at bit-widths ranging from 1.75 to 2.25 bits per weight (bpw).

Technical Methodology and Architecture

According to documentation provided by the ISTA lab, the quantization process employs two distinct methodologies:

  • GSQ (Gumbel-Softmax Quantization): A post-training scalar quantization method designed to learn grid assignments and group scales jointly. The developers claim this approach narrows the performance gap between scalar and vector quantization at low bit-widths.
  • RCO (Riemannian Constrained Optimization): A method that uses gradient descent on task loss to enforce exact bit-width budgets. This allows for different quantization types across the model’s dense path and routed experts.

The release includes three specific builds: IQ1_M (1.75 bpw), IQ2_XXS (2.00 bpw), and IQ2_XS (2.25 bpw). These models require a shared 110.6 GB Engram file, which contains the model’s lookup tables at IQ4_NL precision. While the models are currently incompatible with llama.cpp, they can be run using Strata, a tool designed to serve the models on a single NVIDIA GPU with 12 GB of VRAM.

Benchmark Results and Performance Claims

The ISTA lab reports that the IQ2_XXS build, at 2.00 bpw, matches the performance of the base model across several evaluated benchmarks. Specifically, the lab reports a task average of 93.54, achieving 100% of the base model’s score on benchmarks including AIME25, GPQA-Diamond, and LiveCodeBench v6. The IQ1_M build (1.75 bpw) reportedly retains 98.7% of the base model’s task average.

These findings have met with varying levels of skepticism on community platforms such as Reddit. Some users have questioned the validity of the results, noting that the benchmarks used may be “saturated,” meaning the scores are already near the maximum possible, which could mask the actual degradation caused by quantization. Critics have suggested that the high scores might reflect the model’s ability to prioritize accuracy for specific benchmark-related activations rather than general-purpose intelligence.

User Observations and Limitations

Initial user testing reported on Reddit provides a mixed view of the practical utility of these models. One user reported that the IQ2_XXS build performed significantly better than previous Q2 quantization methods, comparing its quality to a Q3.5. However, they noted that the output was slightly less detailed when tasked with complex coding projects.

Other users reported performance metrics, noting that while the model is capable of running on consumer-grade hardware, such as a single GPU with 64GB of RAM, the generation speed is relatively slow, estimated at approximately 15-20 tokens per second.