Benchmarking Results for DeepSeek V4 Flash

Recent technical benchmarks of the DeepSeek V4 Flash model have provided new data regarding its performance across various hardware configurations, specifically focusing on the impact of speculative decoding and memory management.

Performance on Unified-Memory APUs

Data shared by independent testers on Reddit highlights the performance of DeepSeek V4 Flash 0731 on the Strix Halo architecture (Ryzen AI MAX+ 395). Using the Vulkan backend via llama.cpp, the model achieved a sustained decode speed of 26.76 tokens per second (t/s) at a 4096-token length, with a peak of 35.27 t/s. Prefill speeds were recorded at 236 t/s for a 2209-token prompt.

When utilizing DSpark for speculative decoding, the decode speed on Strix Halo reached 26.76 t/s, compared to 14.2 t/s for raw decode. In comparison, the DGX Spark platform achieved a raw decode of 14.2 t/s but reached 27.3 t/s when using the DSpark engine. While the DGX Spark platform demonstrated significantly higher prefill speeds—reaching approximately 960 t/s—the Strix Halo platform showed a larger relative boost from DSpark, increasing performance by 1.70x compared to the 1.34x boost seen on the DGX Spark.

Optimization Strategies for Single-GPU Setups

In tests conducted on a single NVIDIA RTX PRO 6000 (96GB), researchers reported that DeepSeek V4 Flash 284B can be run by splitting the model between GPU VRAM and system RAM. One configuration, utilizing 19 expert layers in system RAM and the DSpark drafter in RAM, achieved a generation speed of 31.16 t/s during a multi-turn coding workload.

The testing revealed several technical nuances regarding memory allocation:

  • Drafter Placement: Moving the 10.15GB DSpark drafter from VRAM to system RAM allowed for three additional target-model expert layers to be stored on the GPU, resulting in a 4.4% increase in generation speed.
  • Speculative Guessing: The optimal performance was found at three speculative guesses, yielding 33.40 t/s with a 74.7% acceptance rate.
  • KV Cache Quantization: Using a q8_0 KV cache appeared to maintain high performance even as context increased. Decode speeds remained stable between 256K and 768K tokens, with a slight drop to 29.22 t/s at the 1M token mark.

Technical Observations and Limitations

Independent reports have identified several technical requirements for optimal performance. For the DSpark engine, the draft model must be maintained in bf16 precision; testing indicated that using a Q2_K-Q8 quantized drafter resulted in immediate crashes due to invalid token IDs.

Additionally, the use of q8_0 KV cache was found to be more efficient for long-form generation than f16, as it halves the KV load per step and doubles the available context window. On the Strix Halo platform, the Vulkan backend was reported to outperform the ROCm backend in both prefill and decode speeds.

While these benchmarks provide insight into the model’s capabilities on consumer and professional-grade hardware, these results are based on self-published technical reports and have not been independently verified by third-party auditing firms.