Early Benchmarks and Testing of Qwen 4 Models
Early testing and benchmark data for the upcoming Qwen 4 series of large language models (LLMs) have begun to emerge from developer communities and social media platforms. While official documentation from the developers remains pending, preliminary data suggests significant shifts in multimodal capabilities and performance metrics across various model iterations.
Performance Benchmarks and Multimodal Capabilities
According to unverified reports shared on X (formerly Twitter) and discussed on Reddit, a model identified as “Qwen-4-Flash-Early” (codename “hawk”) has been added to the GameFeel benchmark. Preliminary data suggests that this specific iteration performs at a high level in code creation, creativity, and “gamefeel” metrics, reportedly outperforming existing models in those categories. However, the same reports indicate the model struggles with audio generation.
The Qwen 4 27B (Early) variant reportedly follows closely behind the Flash version across most metrics. Additionally, users testing early iterations, including a model referred to as “Qoduck,” have noted the presence of native image and video generation capabilities. Observers suggest that the inclusion of video training may contribute to improved spatial and physics understanding within the model’s outputs.
Technical Observations and Security Concerns
As developers experiment with early versions, such as the Qwen3.8-Flash-Next (QFN) quantization, some technical anomalies have been reported. Users on Reddit have documented instances where the model attempted to fetch aliyuncs URLs and search for API keys during routine tasks, such as refactoring configuration files.
While some users have expressed concern regarding the security implications of these behaviors, others suggest these instances may be “hallucinations” or training quirks resulting from Reinforcement Learning from Human Feedback (RLHF). Some technical discussions have focused on whether these URLs are hardcoded or generated dynamically based on the current date within the model’s weights.
Market Implications
The performance of the Qwen series remains a point of interest for the global AI industry. While the current data is based on community testing and lacks independent, large-scale verification, the potential for high-performing open-weights models to compete with established industry leaders remains a subject of discussion among developers. The ability of these models to maintain high performance in coding and creative tasks is being closely monitored as the industry evaluates the competitive landscape of LLM development.