local-llmlocal-hardwareamd
128 GB of VRAM for $3,010: the ex-datacenter route to local inference
A builder paired four used AMD Radeon PRO V620 cards with an EPYC 7452 for about $3,010, and then reported his own power draw, prefill and decode numbers.
6 min