128 GB of VRAM for $3,010: the ex-datacenter route to local inference

A used-hardware build posted to r/LocalLLaMA this week fits 128 GB of VRAM and 256 GB of system memory around a single server CPU for about $3,010 — four ex-datacenter AMD Radeon PRO V620 cards and an EPYC 7452. On September 13, u/Thin_Pollution8843 opened the thread with "I finished my home inference server," then did what most build logs skip: he published power draw, prefill and decode numbers instead of a screenshot of a model loading. The thread drew roughly 780 points and more than 220 comments, figures worth reading as approximate. The real question is what that older silicon buys that a single modern card cannot.

What $3,010 buys in 2026

The parts list reads like a liquidation lot, which is the entire pitch:

Part	Key spec	Cost
GPUs	4× AMD Radeon PRO V620, 32 GB GDDR6 and 512 GB/s each	$1,400
System memory	256 GB DDR4-2666 RDIMM	$610
Motherboard	Huananzhi D12D	$410
CPU	AMD EPYC 7452 (Zen 2)	$170
PSU	ASRock 1600 W	$220
Case, fans, misc		~$200
Total		~$3,010

The build reuses a Samsung 970 EVO he already had. Three platform details explain why the rest holds together. Each V620 carries 32 GB of GDDR6 at 512 GB/s, a 300 W board rating and a passive heatsink, so server-grade airflow is mandatory. Four cards at PCIe 4.0 x16 want 64 lanes, well inside the 128 the EPYC 7002 platform offers, so nothing lands in an electrically x4 slot. And the platform's eight DDR4 channels are what make 256 GB of RDIMM memory a $610 line item.

He tried a Lenovo P620 workstation first, returned it, and complained about "a ton of proprietary Lenovo shit." A used OEM box is cheap until a firmware lock says no.

$11 per gigabyte, and what it leaves out

Divide GPU spend by capacity and the pitch is obvious: $1,400 for 128 GB works out to about $11 per gigabyte of VRAM. The 96 GB Nvidia RTX PRO 6000 in the 384 GB basement build we covered on September 7 sells on its own for roughly $16,000, or about $167 per gigabyte. That is a gap of more than fifteen times, and it is the whole argument for ex-datacenter hardware in one number.

What the number hides is everything the silicon is not. The V620 is RDNA 2 on a 7 nm process node, a 2020-era design shipped into servers in 2023, and it predates the low-precision matrix paths — FP8 in particular — that current accelerators lean on for quantized weights. It also arrives with no forward roadmap: nobody tunes kernels for a card the secondary market has finished absorbing. Four of them is a bet that community forks keep moving faster than the hardware ages.

The numbers he reported

His headline result, quoted verbatim: "1.3k prefill and 70 tg code/60 tg prose on 128k+ context with MTP-2 on vllm fork." In plain terms, roughly 1,300 tokens per second of prefill and 70 tokens per second of generation on code, 60 on prose, at contexts past 128k, using multi-token prediction with two speculative draft tokens on a vLLM fork. Power, also his: 700-900 W during prefill, 500-600 W during decode, on a Qwen3.8-Next-Flash build quantized to W4A16 with AutoRound — four-bit weights, 16-bit activations.

Every figure comes from one machine, measured by its owner, with no second opinion and no upstream configuration. The code-versus-prose split is the clearest sign that decode speed is not a property of the hardware: speculative decoding only pays when draft tokens are accepted, and code accepts more often than prose. Change the draft model, the speculative depth or the batch size and 70 becomes something else. Treat it as a setting, not a spec. He is candid about the arc: "I was disappointed with this machine and qwen3.8-27b speeds at first. But since Qwen3.8 next running good on it - I'm satisfied."

What the run does not cover

Idle draw was never reported. Neither was prefill of a real prompt set at long context, which matters because a 128k prompt is dominated by time to first token — a figure the post never gives, leaving the 1.3k number shapeless. There is no head-to-head against a single modern card of similar total price, the only test that separates "128 GB unlocks bigger models" from "this is slower per dollar of throughput." And "it runs" is doing less work than it appears: MTP-2 lives in a vLLM fork and the weights are an AutoRound quant, so the recipe is a specific commit plus a specific checkpoint, not a supported path. Anyone expecting an upstream install to reproduce this should budget a weekend of debugging. For why capacity, not model size, is now the binding constraint, our September 12 piece on DeepSeek's 510 GB Flash makes the same case from the model side.

The power bill is the subscription

Hardware is a one-time cost; watts recur. Assume a US residential rate of about 18 cents per kilowatt-hour as a stand-in for his — an assumption about the bill, not a fact about it. At the middle of his decode band, 550 W, eight hours a day is 4.4 kWh: about 81 cents a day, $25 a month. Left serving around the clock, the same 550 W becomes 13.2 kWh a day, or $2.42 — roughly $73 a month. The 900 W prefill ceiling costs more per hour but cannot run continuously, and idle draw was never measured, which matters because a machine that is warm on standby still bills you.

Conclusions

Read as a cost-cutting exercise, this build repeats the lesson of the 384 GB rig: no amount of used enterprise hardware makes local inference cheaper than the API bill you were avoiding. Read as a capacity purchase, it is a genuinely different proposition, because $11 per gigabyte of VRAM is about as low as a hobbyist can reach without renting.

He bought memory and accepted the tax on everything else — older silicon, a fork instead of an engine, a running cost measured in dollars per day. That trade is defensible for someone who wants a large model resident at home and does not mind maintaining the stack, and poor for anyone who wants a supported pipeline and treats debugging as a cost. The next useful data points would be idle draw and a time-to-first-token curve at 128k; until then this is a well-measured anecdote from one machine, which is still more than most build posts supply.