New Inference Engine Targets Apple Silicon Performance

Inco AI has released “Inco Splash,” an open-source inference engine specifically optimized for Apple silicon. According to reports on Reddit, the engine is designed to accelerate large language model (LLM) performance on macOS, specifically targeting users with M3 chips or newer and at least 36 GB of memory.

Unverified performance benchmarks shared by users suggest significant speed increases compared to existing solutions. Claims indicate that Splash can achieve up to three times the decode speed of Ollama and twice the speed of oMLX. In scenarios involving agentic workflows where a primary agent fans out into multiple sub-agents, users report performance gains of nearly four times.

One specific benchmark noted on Reddit shows the Qwen3.8-27B model running at 144 tokens per second on an M5 Max MacBook Pro. However, some community members have raised questions regarding these figures, noting that the results may depend heavily on quantization methods used. Some users expressed skepticism regarding the quality of 4-bit quantization for coding tasks and questioned the technical specifics of the reported speed increases over oMLX.

The engine is available for installation via Homebrew and can be integrated into various agentic frameworks, including Claude Code, OpenCode, Codex, and Hermes. For users preferring a graphical interface, Splash is also available within LM Studio.

As local AI capabilities expand, the hardware required to run them remains a point of contention for enthusiasts. Discussions on Reddit highlight a growing interest in mini PCs, such as the Strix Halo, as portable AI machines. Proponents suggest these devices offer a more accessible entry point for local AI than dedicated servers, which require complex setups and high power consumption.

While high-end hardware like the Mac Studio remains a staple for the Apple ecosystem, some users are opting for alternative configurations, such as custom PCs equipped with NVIDIA RTX 3090 GPUs, citing better value and the ability to run Windows for professional software like AutoCAD.

Global Competition in AI Infrastructure

The landscape of AI hardware continues to shift as international competition intensifies. Huawei has announced several developments aimed at establishing an independent AI infrastructure stack. The company has unveiled the CloudMatrix 384 server as an alternative to NVIDIA’s infrastructure and has introduced “SuperPod” technology, which reportedly supports linking up to 15,488 graphic cards using Huawei’s Ascend-branded chips.

Reports indicate that Huawei is also developing specialized hardware to manage massive data loads, including an AI SSD designed to handle 30 trillion daily tokens. Furthermore, the company is working toward memory independence, with plans to feature in-house High Bandwidth Memory (HBM) in its Ascend 950 chips by 2026.

In the consumer sector, the iPhone 18 Pro has been noted for its integration of on-device AI, driven by the A20 Pro chip. Technical reviews from HTX Studio suggest that these hardware shifts and software updates are increasingly focused on how on-device AI impacts daily usability and battery performance.