Tencent's 200GB Hy4-preview: a 7x compression claim that needs a second look

Tencent's Hy team says it compressed Hy4-preview — its 770B-parameter, 49B-active Mixture-of-Experts flagship — from a 1.5TB BF16 checkpoint to roughly 200GB of GGUF, with the accuracy cost pitched as a rounding error. The announcement, posted on X on August 29 and named MIX-STQ1_0, drew a r/LocalLLaMA thread of roughly 800 points and more than 120 comments. The weights are live on Hugging Face under the AngelSlim org. The benchmark table is short, and the fine print is a pair of llama.cpp patches.

The claim, stated precisely

Hy4-preview itself landed days earlier, on August 27-28: Apache 2.0, 1M-token context, 78 layers, 256 routed experts with top-8 activation, gated DeepSeek-style sparse attention, and a BF16 checkpoint that Hugging Face reports at about 1.56TB. The compression post claims that checkpoint shrinks to roughly 200GiB "and it still works well". The actual artifact is Hy4-preview-STQ1_0.gguf: 213.66 GiB at a 2.38 bits-per-weight average — about 7x smaller. A conventional 4-bit build, Hy4-preview-Q4_K_M.gguf at 435.2 GiB, ships alongside it as the "safe default". The work comes out of AngelSlim, Tencent's open compression toolkit — the same project the Hy4 model card already points to for low-bit quantization — and both GGUFs derive from the Apache-2.0 base.

MIX-STQ1_0: where the bits go

The trick, per Tencent, is "not just going low, it's deciding where": calibration data picks each layer's bit-width. The routed-expert gate and up projections — 97.7% of all parameters — run at 1.3125 bpw in a format called STQ1_0 on 29 layers and at 2.0625 bpw (IQ2_XXS) on the other 48. STQ1_0 is a sparse-ternary format that originates in a llama.cpp pull request: weights are , one of every four lanes forced to zero (3:4 sparsity), a shared scale per 256 weights. Tencent's contribution is the encoder — a weighted least-squares scale and imatrix-aware zero placement — which it says cuts weighted squared error by roughly 90% versus the upstream quantizer on 1,200 expert rows. That claim is at least checkable: the recipe files are published.

Four benchmarks, and the gaps between them

The accuracy story rests on four vendor-reported deltas against BF16:

• MCP Atlas (agentic tool use): 83.7 → 83.2
• SWE-Bench Multilingual (coding): 82.9 → 81.3
• MRCR (multi-turn chat): 81.3 → 81.1
• IFBench (instruction following): 73.5 → 72.5

Retention runs from 98.1% to 99.8% — the source of the "about 98% performance" in the Reddit title. These are unaudited vendor measurements: single runs, no error bars, no third-party reproduction as of this writing. Give Tencent credit for the mix it did choose: agentic tool use, coding, chat, and instruction following are exactly the workloads a 200GB local build would be bought for, and none of them collapsed. The worst erosion lands where it matters most — SWE-Bench Multilingual, down 1.6 points — though 81.3 is still a usable coding score.

More telling is what is absent. The model advertises 1M-token context; none of the four benchmarks exercises anything close. No knowledge or math suites (MMLU, GSM8K), no long-context tasks, no multimodal checks despite the chat template carrying image and video tokens, and no real agentic coding sweeps. A 7x-compressed coding model evaluated on short prompts is an unfinished story.

What r/LocalLLaMA made of it

Reaction split between delight and suspicion. "That is just freaking amazing!", one commenter wrote. Another floated a darker reading: "I wouldn't be surprised if they trained the model in 2-bit, and then released the upscaled version of bf16 first" — a hypothesis with no evidence, but a fair reminder that vendor compression demos have overpromised before. Several commenters also grumbled that the thread linked a screenshot of the X post rather than the post itself, which sits behind X's login wall — friction that only matters because the claim is otherwise self-reported. The sharpest note came from X: "nobody was loading 1.5tb. The product was always whether 200gb still codes." Whether the 2.38 bpw build can actually code — not its score on four curated snapshots — is the product. The download counts hint at real demand: more than 26,000 for the GGUF repo in its first days, versus about 2,100 for the official BF16 checkpoint, and the X post itself drew about 1,600 likes.

The catch: not a drop-in GGUF

"Neither file runs on stock llama.cpp. The hyv4 architecture is not upstream. Apply the patches" — that is the README's first warning. Running the 200GB build means building llama.cpp from a pinned commit with two patch files, one adding the STQ1_0 quant and its CUDA kernels. Full residency wants roughly 214 GiB of VRAM — beyond any single consumer card, but within reach of a multi-GPU workstation — and Tencent measured 204.56 t/s prefill and 20.47 t/s decode on 8x H20. So "commodity hardware" is relative: a 200GB file you can download, but a runtime that is a fork, not a release. The extreme-compression direction itself is familiar — Google's TurboQuant pushed KV-cache compression toward 2.5 bits in March — Tencent is simply attacking the weights of a much larger model at a similar bit depth. For most users, the Q4_K_M build at 435 GiB is the rational choice until independent numbers exist; the 200GB bet only pays off if the 1-2 bit build holds up in real workloads.

What would settle this

Stock or near-stock llama.cpp support once hyv4 lands upstream; independent evals with long-context, math, and real agentic sweeps; error bars; and the community's own imatrix runs, which the published recipes make possible. A fair test is not hard to design: the same prompts at 8K and 128K context, the same agent loop, run side by side on the BF16, Q4_K_M, and STQ1_0 builds. Nobody has published that comparison yet. Until someone does, "kept about 98% performance" should be read as what it is: an unaudited vendor measurement of four benchmarks, on a build that needs patches to run.