Performance Benchmarks in Local AI Inference

New performance metrics shared on Reddit suggest significant advancements in local large language model (LLM) inference speeds when utilizing the “Strata” engine. According to a report from a user testing the software, the engine achieved a token generation (TG) speed of 51 t/s and a prompt processing (PP) speed of 1500 t/s using the Qwen3.8-Flash-Next-GSQ-RCO-GGUF model (IQ3_XXS quantization).

The tests were conducted on a laptop equipped with an Intel 275HX CPU, 64GB of DDR5 RAM, and an NVIDIA 5070ti with 12GB of VRAM. The user noted that at a 43k context depth, the Strata engine outperformed the stock llama.cpp implementation, which reportedly reached only 23 t/s under identical conditions. Furthermore, the Strata engine demonstrated a prompt processing speed of 1500 t/s for a 32k context read, whereas the same quantization in llama.cpp reached approximately 100 t/s.

The Strata engine, currently hosted on GitHub by developer Niko1221, is reportedly optimized for NVIDIA hardware, with experimental support for AMD. The user noted that the engine is specifically designed for select GGUF models, including those from ISTA-DASLab. While the reported speeds are substantial, these metrics are based on individual user testing and have not been independently verified by third-party benchmarks.

The Expanding Landscape of Chinese AI Development

The developments in local inference efficiency occur alongside a broader expansion of the Chinese AI ecosystem. Alibaba has released Qwen3, an open-source hybrid reasoning model, which has reportedly reached 700 million downloads according to Chinadailyhk.com. Additionally, Alibaba’s RynnBrain has been noted for its performance in robotics benchmarks.

In the sector of video generation, Kuaishou Technology has unveiled a new AI video generator intended to compete with OpenAI’s Sora. Meanwhile, ByteDance has released Seedream 4.0, an AI image model.

The scale of Chinese AI infrastructure continues to grow despite ongoing U.S. chip restrictions and talent bans. Reports indicate that Baidu has successfully “illuminated” a cluster of 30,000 self-developed P800 Kunlun chips to train models similar to DeepSeek. Furthermore, China has launched its largest AI science hub in two months, utilizing a cluster of 60,000 chips without the use of U.S. components.

Specialized AI Applications and Research

Research into specialized AI applications is accelerating across various scientific and industrial sectors in China. Recent developments include:

  • Healthcare and Biology: Researchers at Tsinghua University have developed DrugCLIP, a platform designed to accelerate drug screening. Additionally, an AI-based screening system has been developed to identify ocular fundus diseases.
  • Environmental and Space Science: The National Marine Environmental Forecasting Center launched OceanDS for marine forecasting, and researchers have released the first AI large language model dedicated to solar observation.
  • Industrial and Manufacturing: AI adoption in Chinese manufacturing reached 34.2% in the first half of 2026. Specific applications include the use of AI in industrial fermentation processes and the development of QiMeng, an automated processor chip design system.

As the global AI race continues, industry analysts note that China’s models are seeing significant adoption, with reports indicating Chinese AI models accounted for 4.69 trillion weekly tokens.