New Benchmark Data Suggests Narrowing Gap in AI Development
New data released via the open-source AI leaderboard whatmodel.sh suggests that the performance gap between China’s open-weight models and the United States’ closed-source models has narrowed significantly. According to the platform, which aggregates massive datasets of published benchmark results, China’s open-weight models now trail US closed models by a margin of approximately seven months.
The data was compiled by developer Jake Manger, who launched the whatmodel.sh platform to provide a transparent view of model performance. The dataset includes 18,755 results for 1,211 models across 617 benchmarks. The platform distinguishes between 14,436 results from independent evaluations—such as papers from the ACL Anthology and arXiv, Epoch AI, and the LMArena dataset—and 7,677 scores reported directly by the respective labs.
Methodology and Data Transparency
The developer has made the dataset, released under a CC BY-SA 4.0 license, and the scoring code, released under the MIT license, available on GitHub. The platform aims to provide a traceable link for every number, connecting each score back to its original published source.
While the data provides a granular look at the current state of the AI race, it is important to note that much of the information is derived from self-reported lab data or independent academic papers, and the specific claim regarding the “seven-month” gap is a conclusion drawn from the comparative analysis of these benchmarks rather than a single official industry report.
Implications for the Open-Weight vs. Closed-Model Market
The findings have sparked discussion regarding the competitive landscape between proprietary models, such as those from Anthropic and OpenAI, and open-weight alternatives, many of which originate from Chinese developers.
In online discussions following the release, some users noted that the rapid advancement of open-weight models may pressure the pricing and necessity of high-cost monthly subscriptions for closed models. For instance, while some users reported that closed models like Anthropic’s Opus 5.5 remain superior for complex programming tasks involving multiple interacting systems, others suggested that local, open-weight models are increasingly sufficient for smaller scripts and specific creative tasks like image or video generation.
The data highlights a trend where the capabilities of open-weight models are rapidly approaching the performance levels of the most advanced proprietary systems, potentially altering the economic incentives for developers and end-users in the AI ecosystem.