Localized AI Deployment on Legacy Hardware
A new open-source project has demonstrated the ability to run the Qwen3.5-9B large language model (LLM) on a standard PlayStation 4 (PS4) console. According to a developer’s report on Reddit, the implementation allows the hardware to process text, listen, and write code entirely offline, without the need for cloud services, subscriptions, or a Linux operating system.
The project, hosted on GitHub under the name “garlic-llm,” utilizes the PS4’s internal GPU to drive the model. To accommodate the hardware’s memory constraints, the model weights have been re-packed into custom 4-bit and 3-bit formats, reducing the footprint to approximately 4.5 GiB.
Performance Benchmarks and Technical Implementation
Performance metrics shared by the developer indicate that the model operates at speeds between 31 and 39 tokens per second (tok/s) on a stock PS4. A significant factor in these speeds is the use of the MTP head included with Qwen3.5, which facilitates speculative decoding. By drafting up to two tokens that the full model then verifies in a single pass, the developer claims the processing speed increased from 22 tok/s to the current 31–39 tok/s range.
While the results demonstrate the utility of legacy gaming hardware for local AI tasks, some users in the developer community have offered technical caveats. One commentator noted that while the 9B model is efficient for tasks such as Optical Character Recognition (OCR) and data extraction, it may lack the complexity required for autonomous “agentic loops,” suggesting that larger models, such as the Qwen3.8-27b, would be more appropriate for such functions.
Scalability and Hardware Limitations
The technical discussion surrounding this development suggests that the capabilities of Sony’s older hardware may extend further depending on the software environment. Some users noted that the PlayStation 5 (PS5) and the BC-250 can run the larger Qwen3.8-27b model using IQ4_XS quantization, though this requires a headless setup and a custom distribution.
As local AI tools continue to evolve, developers are increasingly exploring ways to utilize existing consumer hardware. While the “garlic-llm” project provides a functional proof-of-concept for offline, local execution on a PS4, the practical application for high-level reasoning remains a subject of ongoing debate among users.
