High-Performance LLM Benchmarks Reveal Efficiency Gains on Consumer Hardware

New performance benchmarks for large language models (LLMs) have emerged, highlighting significant inference speeds on high-end consumer silicon. Recent data shared by developers on Reddit provides a technical look at how models such as GLM 5.3 Flash and Qwen3.8-Flash-Next perform on Apple’s M5 Ultra and AMD’s Strix Halo architectures.

GLM 5.3 Flash Performance on Apple M5 Ultra

According to reports from users on Reddit, the GLM 5.3 Flash model achieved a decoding speed of 68.8 tokens per second (tok/s) when running on an M5 Ultra 256GB system. The test was conducted using the oQ4e+MTP quantization method on oMLX 0.7.0.

The benchmark recorded a prefill rate of 1,878 tokens. While the results demonstrate the high throughput of the M5 Ultra, some community observers noted that the single-stream performance appeared to hit a ceiling, leading to discussions regarding the efficiency of the architecture under concurrent workloads.

Qwen3.8-Flash-Next and the Strix Halo Platform

In a separate development, developers from Yamz Labs have released performance data for Qwen3.8-Flash-Next (a 125B Mixture-of-Experts model with 6B active parameters) running on an AMD Strix Halo mini PC equipped with a Ryzen AI Max+ 395 processor and 128 GB of memory.

The developers released the 95 GB EXL3 weights alongside a new open-source inference engine named Kyojin, built on ExLlamaV3. The reported performance metrics include:

  • Decoding Speed: 44 to 59 tok/s when utilizing speculative decoding, depending on the specific task (approximately 47 tok/s for chat and 58 tok/s for coding). Without speculative decoding, the speed dropped to 32.7 tok/s.
  • Prefill Rate: The engine maintained a relatively flat prefill rate across various context lengths, ranging from 1,412 tok/s at 4K context to 1,367 tok/s at 128K context.
  • Fidelity: The model achieved a 94.1% top-1 agreement with the original FP8 model.

The developers noted that while their engine shows higher fidelity to the original model compared to competitors like Halogen, Halogen currently maintains a slight advantage in raw speed. Yamz Labs stated that they are currently reworking the core of the Kyojin engine to close this performance gap.

Comparative Context and Hardware Implications

The benchmarks highlight the increasing capability of consumer-grade hardware to handle large-scale models. For comparison, a user of the llama.cpp engine reported lower performance on the same Strix Halo hardware, achieving approximately 30 tok/s with speculation and 500 tok/s prefill.

The data underscores the competitive landscape between Apple’s silicon and AMD’s Ryzen AI Max+ series in the local LLM inference market. While these results are based on self-reported benchmarks from developers and community members, they provide a technical baseline for the efficiency of MoE (Mixture-of-Experts) architectures on high-memory consumer devices.