Technical Development in Distributed Computing

A developer has demonstrated a method to utilize an iPhone as a secondary GPU to augment the processing power of a MacBook, according to reports from a recent discussion on Reddit. The setup, which utilizes a 10 Gb/s USB-C connection and custom software, aims to alleviate memory and processing constraints when running large language models (LLMs).

The developer, who has released the project’s code and benchmark scripts via GitHub, tested the configuration using a 24 GB M4 Pro MacBook and an iPhone 17 Pro Max. The process involves the Mac executing the initial layers of a model and streaming activations to the iPhone, which then processes subsequent layers using its A19 Pro GPU.

Performance Benchmarks

According to the unverified benchmarks provided by the developer, the integration of the iPhone’s silicon resulted in significant increases in prefilling speeds for the Qwen 3.8 27B model. The reported performance gains vary depending on the context window size:

  • 8k context: 177 tokens per second (tok/s) compared to 132 tok/s on the Mac alone (+35%).
  • 16k context: 157 tok/s compared to 109 tok/s on the Mac alone (+44%).
  • 32k context: 130 tok/s compared to 101 tok/s on the Mac alone (+29%).
  • 48k context: 113 tok/s compared to 87 tok/s on the Mac alone (+30%).

In a specific test involving a 27k-token agent session, the developer reported a reduction in processing time from 245 seconds on stock llama.cpp to 168 seconds using the modified setup.

Memory Management and Context Windows

The setup also addresses memory limitations. When the context window exceeds 64k tokens, the developer’s implementation shifts certain tasks to the iPhone. The phone manages the oldest KV (Key-Value) pages, allowing the Mac to maintain a stable memory footprint.

The developer noted that the iPhone’s Neural Engine was utilized to compile 16k-key pages of old context into a model, which reportedly reduced writing time from 279 ms to 176 ms per token at a 140k context level. This distributed approach allowed for a growing session of up to 128k tokens at 8-bit precision, a feat the developer claims would otherwise require dropping to 4-bit context on the Mac alone.

Limitations and Future Outlook

The current implementation does not increase writing speeds for context windows below 64k, as those tasks remain handled by the MacBook. Additionally, the phone currently only joins prefills for approximately 83% of tokens in real-world sessions.

While the results suggest potential for distributed AI processing, the developer noted that further work is required to optimize the setup for newer model architectures, such as the Qwen 4 preview. The project remains in a developmental stage, with the developer expressing interest in testing the setup on future hardware, such as the iPhone 18 Pro Max.