Developer Leonardo Alegría has released Garlic LLM, an open-source project that enables a jailbroken PlayStation 4 to run the 9-billion-parameter Qwen3.5-9B AI model entirely offline. The application bypasses standard system drivers to communicate directly with the console’s 2013-era GPU, delivering text generation speeds of up to 43 tokens per second.
Direct GPU Communication on Legacy Hardware
The project relies on GoldHEN, a homebrew enabler for exploited consoles, to function. Unlike previous attempts to run large language models on PlayStation hardware, which often utilized driver layers like PS5_Vulkan, Garlic LLM operates without them. The application uses OpenCL C compute kernels compiled with LLVM for the AMD Graphics Core Next architecture. These kernels handle the heavy mathematical lifting, such as matrix multiplications, while a plain C program on the CPU manages the tokenizer, interface, and audio processing.
This direct approach allows the model to fit within the console’s 8GB of shared memory. The 9-billion-parameter model is compressed using a combination of 4-bit and 3-bit weights, reducing its size to approximately 4.5GiB. Additional memory is allocated for the attention cache and speech models, bringing the total package size to 5.66GB. The system supports a 16K-token context window, a reduction from the model’s native 256K, but sufficient for most local interactions.
Performance and Voice Capabilities
Alegría credits the project’s speed to the multi-token prediction head built into the Qwen3.5 architecture. The system drafts up to two tokens ahead and verifies them with the full model, a technique that significantly boosts performance on the aging hardware. In text chat, the model runs at 31 to 39 tokens per second, while code generation can reach up to 43 tokens per second. A 300-token response typically takes under nine seconds to generate.
Beyond text, the setup supports voice interaction. It uses the Chatterbox speech model to provide spoken replies in seven languages at approximately 22 tokens per second. Users can also input dictation via Nvidia’s Parakeet TDT 0.6B v3 model, which supports 25 European languages. The default temperature is set to 0.7 for general tasks, though the developer notes that a temperature of 0 can be used for more predictable outputs.
The project is available as a single download on Hugging Face and can be installed via the PS4’s Package Installer. It requires about 6GB of free space, or 12GB if the package is copied to the internal disk. Alegría tested the software on a base and Slim PS4 running firmware 11.52. The PS5 is not supported, as the code is specific to the PS4’s system software, and the PS4 Pro has not yet been tested.
Source: Tom’s Hardware

