Hy4 flash sells cheap tasks, and 64 billion of its parameters are a lookup table
Tencent's Hy4 flash keeps 220B parameters in memory and touches 12B per token, at $0.281 per million out. The open weights make the cost claims testable.

Tencent published Hy4 flash as an open-weight model on 16 November, leading with two numbers that are easy to merge into one: 224 billion parameters total, 12 billion active per token. They are different measurements: the memory you must hold, and the arithmetic you pay per token. The third number is the only one a buyer can check without trusting anybody: $0.070 per million input tokens, $0.281 per million output, $0.003 cached.
The checkpoint is smaller than the post says: Hugging Face's safetensors index counts 220,251,201,958 parameters, split by the card into 155.5 billion of language backbone, 63.5 billion of Engram tables, 0.43 billion of vision encoder and 0.85 billion of drafter. Rounding a memory figure up by four billion is harmless. Reading it as a dense model of that size is not.
Memory and arithmetic are two different invoices
Mixture-of-experts models separate weights from compute; this one separates them further than most. Sixty-four billion of the parameters are Engram tables: learned bigram and trigram representations retrieved by multi-head hash lookup at layers 1, 11, 21 and 31, eight hash heads per n-gram order and 10.3 million slots per head. Nothing in that path multiplies a matrix through the forward pass. It reads a table. So "224B" is a memory-bandwidth claim first, and comparing it to a dense model of the same size gets both bills wrong: the dense model would spend eighteen times the arithmetic per token and occupy roughly the same bytes on disk.
An Engram table cannot be pruned without losing the lookups it stores, which is why the Hy4-preview compression effort became a fight over a 1.5TB checkpoint rather than over FLOPs, and why the card ships a GGUF build through AngelSlim instead of pretending a four-bit copy is free.
What the design does buy is a smaller cache. The 42-layer backbone runs sparse attention alongside a 512-token sliding window on 30 of its layers, and the sparse layers select at most 2,048 positions per token through MLA. Thirty short windows keep the KV cache from growing the way full attention's does, which is where the claim of one twenty-fourth of Hy3 and one sixth of Hy4 comes from.
The price sheet, and what the market charges
| Per million tokens | Cached input | Input | Output |
|---|---|---|---|
| Hy4 flash | $0.003 | $0.070 | $0.281 |
| Hy4 | $0.042 | $0.834 | $2.501 |
That is 8.4% of its flagship's input rate and 11.2% of its output rate: aggressive, not implausible, since the arithmetic per token is roughly a sixth and the cache a sixth of the size. On OpenRouter one provider serves it, Tencent Cloud itself, at half price through 30 November: $0.035 in, $0.1405 out, $0.0015 cached. The router's weighted average, $0.00609 per million input at a 95.4% cache hit rate, says the traffic is mostly repetition and the cache line is doing the work.
Throughput there reads 61 tokens per second at the median, with 3.1 seconds of latency, against Tencent's claim of past 150. Both can hold: the higher figure depends on megakernels, four-bit weights and the DFly2 drafter on a tuned stack. The 61 is what an API customer gets, and it belongs in a capacity plan.
WorkIndex is Tencent's ruler, and Tencent holds it
The headline comparison is 42.1 on WorkIndex against 39.7 for Hy4 preview, rendered as "slightly above Hy4 preview in overall task performance." Accurate, and modest: the shipping Hy4 scores 51.4 on the same internal scale, so the cheap model gives up 9.3 points to the flagship it replaces for everyday work; Tencent steers long-horizon software engineering back to Hy4.
Then come the ratios. Average cost per task is one thirteenth of Hy4 preview, one twentieth of Hy4, one eighth of DeepSeek-V4.1-Flash-1017 at its Max tier on peak-hour pricing. Three denominators, all chosen by the seller: a preview build, its own flagship, and a rival at the priciest tier in the priciest hour. The token savings are documented, with reinforcement learning halving average end-to-end output, but the multiplier is what a private harness yields against a baseline the buyer did not set. WorkIndex is grounded in WorkBuddy tasks and exists only inside Tencent.
The human check is stronger and still insular. Eighty-five internal WorkBuddy users ran 259 tasks across 27 categories, scoring Hy4 flash 2.79 out of 4 against 2.81 for Claude Haiku 5.5, with the shortest median completion at 6.7 minutes and RMB 0.26 per task. Blind, yes. Independent, no.
No third-party score sits beside this yet: Artificial Analysis has no Hy4 flash profile. The benchmark appendix ships as a single image with no numeric table, so the open-set cells can be read but not rerun.
What the base checkpoint gives away
The most useful table is the least promoted: base checkpoints on standard few-shot tasks, where Hy4 flash Base loses to Hy4 Base on nearly every row.
| Benchmark | Hy4 flash Base | Hy4 Base |
|---|---|---|
| MMLU | 88.7 | 91.6 |
| SuperGPQA | 57.8 | 62.7 |
| SimpleQA | 37.5 | 57.2 |
| MMMU-Pro | 52.0 | 59.6 |
| MBPP-plus | 73.7 | 74.0 |
SimpleQA is the row that matters: the gap is 19.7 points. Engram is supposed to buy factual recall cheaply, hashing short n-grams into a large table instead of spending layers on memorisation. On Tencent's own numbers the compressed model is decisively worse at open-domain factuality, and level only on code. The lookups did not close the knowledge gap; they made a smaller network affordable. Post-training recovers enough for office tasks, which is what WorkIndex measures, but it does not make a compressive design a free lunch.
Open weights, under a licence that is not open source
The real compliment is that the claims are now falsifiable. Weights, an FP8 build and the base checkpoint sit on Hugging Face under a licence tagged only as "other": the Tencent Hy License, Version 1.0, derived from Apache 2.0 with a Section 10 written for model-as-a-service businesses. Run it internally, fine-tune it, quantise it. Offer general inference to third parties and you need a separate written licence once worldwide revenue passes $50 million in any twelve months, or a product carrying it passes 100 million monthly active users. A commercial gate, not copyleft: the difference between open weights and open source. First-week adoption was modest at 159 downloads and 225 likes.
For a team already renting a flash-tier endpoint, the case is plain. Output costs a ninth of Hy4's, the cache is small enough that long documents stop being a tax, the window is a full 1,048,576 tokens, and the weights let you measure the model on your own traffic instead of arguing about WorkIndex. The part to distrust is the cost-per-task ratio, Tencent's number from Tencent's harness against a competitor's worst-case price. Pull the checkpoint, run forty of your own tasks, count the tokens; the licence lets you, and that is the honest reason to take the release seriously.


