A 90M LLM Now Runs on the Sony PSP — at Half a Token Per Second

On September 4, developer u/liright posted something on r/LocalLLaMA that local-AI communities love: a conversational language model running on a gaming handheld from 2004. The thread — roughly 1,030 points and 84 comments — shows a 90M-parameter LLM generating replies entirely on a Sony PSP, the console that launched in Japan in December 2004 and in North America in March 2005. No server, no API: the 333MHz MIPS CPU does every calculation, and the model's weights live on a Memory Stick.

What Happened

The project, LLMPSP by developer thatblend, is a custom C runtime that boots from the PSP homebrew menu and turns the console into a miniature text generator. Inside is Falcon-H1-Tiny-90M-Instruct, an instruction-tuned model from TII, quantized to 4 bits in a custom FHQ4 format designed around the Memory Stick's slow reads, with the int8 KV cache and quantized matvec kernels running on the PSP's VFPU vector unit.

The technical picture:

• Model: Falcon-H1-Tiny-90M-Instruct, 4-bit FHQ4 quant of a hybrid attention + Mamba2 design (24 layers)
• Weights: 52MB file on Memory Stick; ~44MB cached in RAM, the rest streamed per token
• Hardware: PSP-2000/3000/Street/Go with 64MB RAM running custom firmware; the 32MB PSP-1000 is not supported
• Speed: 0.5–0.6 tokens per second on a PSP-3000 — one to three minutes per reply

The author is blunt about it: 90M parameters is "about the max it can do without atrocious inference speeds." The output is the real tell — crappy poems, short stories, code that does not run. The model knows Apple makes macOS, then happily hallucinates the next answer. The README's own "working" Python example even carries an editor's note: "the script did not work." The verdict, in thread and repo alike, is "not really useful for anything, but it's fun."

Why This Matters

This is the opposite end of the local-LLM spectrum from Qwen 3.8 27B at 50 tok/s on a 16GB GPU and Framework's 192GB unified-memory desktop, both covered here last week. The PSP runs 90M parameters at half a token per second, yet it is genuinely local: fully offline, pocket-sized, two decades old. That is the real news — quantization and clever runtimes have shrunk instruction-tuned models until one fits on a memory card. But mind the gap between headline and experience: a chatbot on a PSP sounds like something; waiting one to three minutes per reply feels exactly like 2004.

What's Next

The code is MIT-licensed; the Falcon-H1 weights fall under TII's Falcon license, so check the terms before redistributing. Anyone with a CFW-capable PSP can sideload the runtime, and overclocking the CPU buys a little more speed. Expect more retro-hardware ports in the coming months — and expect them to stay curiosities. The charm is real, the utility is not, and that is fine: a handheld museum piece that can finally talk back is still a museum piece.