Comparative Performance Analysis of Local Mixture-of-Experts Models

A recent independent experiment conducted by a developer on the social media platform Reddit has provided a performance comparison between two prominent Mixture-of-Experts (MoE) models designed for local execution: DeepSeek V4.1 Flash and Qwen 3.8 FN. The test utilized a complex, single-shot prompt requiring the generation of a fully interactive 3D V8 engine simulation using Three.js/WebGL.

While the results offer insights into the capabilities of these models, the findings remain unverified by independent third-party benchmarks and represent a single-instance test case.

DeepSeek V4.1 Flash Performance Metrics

The DeepSeek V4.1 Flash model was tested using an IQ1_M quantization. According to the report, the model generated approximately 250,000 tokens at a rate of 94 tokens per second (tok/s). The process took roughly 60 minutes to complete, utilizing a context window of 346,000 tokens without compaction.

The resulting simulation was described as functional upon the first attempt, with all interactive controls working as intended. However, the developer noted that the model omitted certain smaller internal components in the engine’s cutaway view. The report suggested that DeepSeek V4.1 Flash demonstrated a higher degree of autonomy in problem-solving during the task.

Qwen 3.8 FN Performance Metrics

The Qwen 3.8 FN model, running on NVFP4 quantization, utilized a multi-agent architecture consisting of one main agent and five subagents. This configuration achieved an aggregate throughput of approximately 350 tok/s. The total generation time was approximately 70 minutes, producing 443,000 tokens from the main agent and a total context of over 804,000 tokens, involving two compactions.

The output from Qwen 3.8 FN was more detailed than the DeepSeek counterpart, featuring a more comprehensive cutaway view and simulated engine sounds. However, the developer noted several technical inconsistencies, including “floating parts” and instances where subagents failed to execute promised file-writing tasks.

Technical Observations and Hardware Context

The experiment was conducted on a high-specification hardware rig consisting of an AMD Ryzen 9 9950X processor, 128 GB of DDR5 memory, and dual GPUs (RTX 5090 and RTX 5060 Ti).

In the discussion following the test, users noted the inherent limitations of the multi-agent approach used by Qwen, specifically regarding “narration-only” errors where agents claim to perform actions without executing them. One participant noted that such errors are common in large language model (LLM) decoding.

The comparison highlights a trade-off between the raw speed and complexity of the Qwen multi-agent system and the functional reliability and autonomous reasoning observed in the DeepSeek V4.1 Flash model.