Technical Breakthrough in iPadOS Local AI Inference

LemonSeed Studio has released version 0.5.8 of its iPad-based editor and Integrated Development Environment (IDE), featuring on-device large language model (LLM) inference via an external AMD GPU. According to reports from developers on Reddit, the setup utilizes an iPad Pro connected to a Sapphire Radeon AI PRO R9700 via a Thunderbolt enclosure.

The technical implementation relies on the LemonSeed Engine (LSE), a cross-platform engine designed for Linux, macOS, and iPadOS. To enable hardware acceleration on iPadOS, the software embeds the unmodified upstream Linux amdgpu and amdkfd drivers as a PCIDriverKit extension. This allows the iPad to interface with the desktop-grade hardware to run complex models locally.

Performance Benchmarks and Technical Specifications

In unverified performance tests shared by the developer, the setup was used to run the Qwen3.8-27B model (Q4 quantization) with a 131k context window. The benchmarks, which include the use of Q8 DFlash2 draft tokens and speculative decoding, reported the following results for the R9700 on iPadOS:

  • Decode Speed: 31.2 tokens per second (baseline) and 158.5 tokens per second when utilizing MTP=3 and DFlash2.
  • Prefill Speed (4K tokens): 1,636 tokens per second.
  • Prefill Speed (32K tokens): 1,395 tokens per second.

For comparison, the developer reported that the same R9700 hardware achieved 159.4 tokens per second (decode) on macOS and 144.6 tokens per second on Linux. When tested on the Strix Halo (Radeon 8060S) platform under Linux, the decode speed with DFlash2 was measured at 64.4 tokens per second.

The LemonSeed Engine achieves these speeds by recording the model’s forward pass as a graph, fusing operations, and generating custom kernels for the specific GPU in use. The engine also employs speculative decoding, using MTP and DFlash2 to determine the optimal number of draft tokens to verify based on measured acceptance rates.

Software Updates and Developer Feedback

The 0.5.8 update introduces several technical refinements, including the activation of DFlash2 draft trees by default, which verifies a tree of draft candidates when measured to be faster than a standard chain. The update also includes support for Strix Halo (gfx1151) prefill and decode, featuring fused gate/up GEMM and optimized query tiles for prefill attention. Additionally, the Linux archive now bundles its own HSA runtime, requiring only the amdgpu driver with /dev/kfd access.

While the reported speeds are significant, some community members on Reddit have raised questions regarding the consistency of these figures. Specifically, users have noted that performance for similar models can sometimes degrade as the context window fills, and some have questioned the feasibility of maintaining high token-per-second rates during extended workloads.

The project is currently available via GitHub, with the engine and Studio tools hosted under the developer’s repository.