All News
open-weightslocal-inferencemixture-of-expertsquantizationedge0

Edge0-145B puts 145B parameters in 29GB — and won't name the laptop

Edge0 released a 145B model it says runs on a laptop in 29GB. The weights are real and 105GB large; the hardware in the spec table is a sentence, not a machine.

Vlad MakarovVlad Makarovreviewed and published
6 min read
Edge0-145B puts 145B parameters in 29GB — and won't name the laptop

Edge0, which signs its announcements from AutoArk, published four checkpoints and an inference framework on 11 November and wrote them up the next day. The claim on its blog is that Edge0-145B is "the world's first open-source on-device AI model with 145 billion parameters," running on "a compatible laptop" at 29GB peak memory and more than 80 tokens per second. The Hacker News thread took it to 334 points, mostly arguing over whether the number describes capability or bookkeeping.

Both halves of that argument are answerable now, because the weights are public. The repositories exist and the download is far larger than 29GB. The licence is not an open-source licence, and the project's own documentation describes different hardware at different speeds than the post does.

What the spec table actually says

The performance table has five rows and the hardware column names a category, not a machine. Memory: "29GB peak memory," enabling "a compatible personal laptop, such as a MacBook Pro, without a server." Prefill: "1,100+ tokens/s." Decode: "80+ tokens/s." Power: "approximately 120W typical." Context: 128K-256K at "essentially the same speed on long inputs."

The paragraph beneath it offers the prior release as a comparison and then withdraws it — Edge0-35B at under 2.5GB and 37+ tokens per second on an iPhone 18 Pro Max, followed by "These measurements apply to different models and hardware." The same distance covers the 145B row, which names no GPU, CPU, RAM total, operating system or build. The units drift as well: the table says 29GB, the comparison paragraph says "28GB peak memory," and the benchmark table concludes that 29GB "enables local use on a 48GB laptop."

The download is real, and it is 105GB

Four repositories exist under the Edge0 organisation, all created on 11 November, none gated. The sizes below sum every file in each Hugging Face tree on 14 November, in decimal gigabytes.

RepositoryExpert formatFilesDownload
Edge0-145B-q3kQ3_K93105.4 GB
Edge0-145B-NVFP4NVFP445112.9 GB
Edge0-120B-q3kQ3_K9292.3 GB
Edge0-120B-NVFP4NVFP44497.6 GB

The 145B NVFP4 checkout had more than 14,000 downloads as of 14 November.

This is not GGUF or safetensors in any ordinary sense: a model-skeleton.gguf of 5.99 GB, raw experts.bin files, a ple-fp8.gguf table of 51.2 GB, and separate latent projection files. Hugging Face's parameter counter reads the skeleton and reports 4.95 billion parameters for a repository named 145B — a fair summary of what a standard loader can see.

The licence is other, named edge0-community-model-1.0: personal and research use, modification and commercial use of outputs are allowed, while model sales, paid inference services and "independent commercial coding or office assistants" need separate written permission. The upstream Qwen Community License applies too. Open weights, yes; open source in the OSI sense, no.

Where 29GB comes from

29 billion bytes across 145 billion parameters is 0.2 bytes each, 1.6 bits. No 3-bit or 4-bit format reaches 1.6 bits: 105.4GB across 145 billion parameters is 0.73 bytes per parameter, 5.8 bits.

The tiers the post describes close that gap. The expert pool is 84.6 billion parameters at latent width 1792, down from 2560, so every expert is a 1,630,720-byte record holding 3,440,640 parameters — 3.79 bits each in Q3_K. Those records are experts.bin, 40.08 GB, and the post says "an expert library of approximately 40GB sits in system memory." The 51.2 billion parameter n-gram table ships as ple-fp8.gguf, 51.2 GB, and the post says "approximately 48GB of vocabulary data resides on the SSD and is read on demand" — 47.7 GiB, the same file in different units.

On any given token 10.12 billion of those parameters are read. The project's own documentation puts the Q3_K build at 40.81 GiB of weights, split under memory pressure into 23.14 GiB of VRAM and 31.28 GiB of host RAM — 54.4 GiB total, "disjoint expert sets, nothing counted twice."

So 29GB is not the model's memory. It is a peak on one tier of a pipeline whose other tiers the same post says hold 40GB of experts and 48GB of lookup table. Which tier is unstated, and that is the claim.

The repository disagrees with the announcement

The documentation's requirements section is one line: "Linux or Windows with an NVIDIA GPU and CUDA 13." Its platform table marks macOS as a dash for both models, and its Apple M2 Ultra rows "do not establish macOS release availability." The blog's FAQ says the model "supports platforms like macOS, Windows, and Linux, including Apple M-series chips and NVIDIA RTX platforms." One sentence is wrong, and it isn't the engineers' one.

The speed tables name the machine the announcement doesn't: an RTX 5090 D with 24GB, a desktop card at 575W board power, where 145B Q3_K posts 2,169.9 tokens per second of prefill at 4K, 3,554.9 at 16K, 99.7 chat decode and 121.2 agent decode. NVFP4 on the same card gives 79.6 chat decode, very likely where "80+" comes from.

Faster than 80 tokens per second on a 24GB desktop GPU: documented. Faster than 80 on a laptop drawing 120W: asserted, with no configuration, no build and no log.

The half nobody can check

No independent benchmark existed as of 14 November. Artificial Analysis has no Edge0-145B page — the model URL returns a 404 — and the six scores in the post and the repository are Edge0's internal evaluations, as the fine print says: "Edge0, Qwen, DeepSeek, and Claude results come from Edge0's internal evaluations."

Without the framing it is narrower than "rival frontier cloud models." Edge0-145B beats its own base model on DeepSWE 1.1, 64.6 to 58.7, and Terminal-Bench 2.1, 86.5 to 83.3, then loses on HLE (35.4 to 35.9), GPQA-Diamond (90.4 to 91.7) and IFBench (80.6 to 81.3). It wins Terminal-Bench and IFBench against Claude-Opus-4.6 and loses HLE 35.4 to 40.0. A 72.8 six-benchmark mean against Qwen's 71.7 is a real result and a modest one, and the thing it beats is a 4-bit deployment of its own parent.

What to do with it

The engineering is the part worth reading: latent expert compression, experts streamed from host memory, an embedding table memory-mapped from disk, speculative decoding sized to the weight read rather than the token. Anyone who has watched a 24GB card refuse a 70B model should read the architecture page before the announcement.

What won't survive is the laptop framing without a machine behind it — the gap Apple's Harbor leak left open and October's four releases closed by publishing file sizes. A 105GB download needs 105GB of disk before it needs a laptop, and the licence forbids the commercial coding assistant this model targets without a written agreement. Both are in the README, neither is in the table. The rest is arithmetic: 145B here is 94B of model plus 51B of table, and only one of those has to be in memory.

Related Articles

Scroll down

to load the next article