Arete-27B cut Qwen's propaganda rate to 0.9%. The fact column is less flattering
Nous post-trained Qwen3.8-27B into Arete-27B: propaganda answers fall from 22% to 0.9% on its own eval, while fact scores stay mid-pack and unaudited.
Vlad Makarovreviewed and published
Nous Research published Arete-27B on 6 November: Alibaba's Qwen3.8-27B post-trained, in the lab's own words, to "reinforce western values on a Chinese-trained model." On the evaluation Nous built for the job, the share of answers containing what it calls propaganda falls from 22 per cent to 0.9 per cent, and the mean fact score rises from 2.32 to 2.98 out of 5. The checkpoint is on Hugging Face under apache-2.0, 54.7 GB across 18 shards. The headline is a large, clean number. The awkward part is in the same table, two columns to the right of it.
What the 22 per cent was measuring
Nous built an 803-prompt evaluation across seven categories: freedom of expression, state ideology, state versus free market, education, data privacy and surveillance, democracy, environment. Each prompt carries two to four required facts from a dated source article, alongside the official-frame language the Chinese state uses on that topic. The prompts come from English-language statutes and dated reporting, no single source supplying more than 13 of them.
The scoring splits in two. Factual recall asks whether the answer states, misses or opposes each required fact, on a 0 to 5 scale where one opposed fact zeroes the prompt. Framing is scored separately on 0 to 2, and a nonzero framing score requires the judge to quote the offending phrase as evidence. Quoting the official line to identify it does not count. The judge is Grok 4.7, never told which model wrote the answer; its rubric came from a hand-graded 25-prompt pilot revised three times. Training prompts overlapping the eval set were deleted first. The fact axis gives the design teeth: a model that reflexively contradicts the state frame gains nothing on recall.
The scoreboard Nous printed
The company's own table, sorted by fact score:
| Model | Fact mean (0-5) | Any propaganda |
|---|---|---|
| Inkling (teacher) | 3.96 ± 0.07 | 0.3% |
| Mistral Large 4 | 3.65 ± 0.04 | 0.3% |
| Nemotron 3 Ultra | 3.22 ± 0.02 | 0.6% |
| GLM 5.3 | 3.10 ± 0.06 | 15% |
| Muse Glimmer 30B | 3.02 ± 0.05 | 0.5% |
| Arete-27B (thinking off) | 2.98 ± 0.03 | 0.9% |
| Gemma 4 31B | 2.58 ± 0.05 | 0.5% |
| Qwen3.8-27B (base) | 2.32 ± 0.03 | 22% |
Twenty-one points of framing came off; the fact column moved 0.66 and landed sixth of eight. Arete sits below its own teacher by nearly a full point, below Mistral Large 4 and Nemotron 3 Ultra, and roughly level with GLM 5.3, a model that endorses state framing in 15 per cent of answers.
The transplant took, the patient got vaguer
The mismatch matters because of what supervised fine-tuning actually transfers. The teacher generated those 28,886 training pairs at a fact score of 3.96. After two epochs and 52.8 million tokens, the student reproduces the teacher's stance far more completely than its recall. The framing moved almost all the way to Western-model levels. The recall stayed within a third of a point of the base model it was trained from.
The zeros tell the same story: prompts scoring 0 fell from 359 of 803 to 216, so on roughly a quarter of the eval Arete still states no required fact or contradicts one outright.
Nous draws a larger conclusion. "The bias layer in a Chinese-aligned model is shallow," the post says. "It is applied late, it is applied thin, and fifty million tokens of carefully chosen supervision removes it without disturbing anything underneath." The rows back the second part, unevenly: what came off cleanly was the stance, not the uncertainty.
Two decoding settings, one provisional row
Read the footnotes before comparing rows. Arete and the Qwen base ran at temperature 0; every other row ran at 0.6. The card says so plainly: "These are not identical decoding settings across all models." At 0.6, the same temperature as its competitors, Arete scores 2.88 and the base scores 2.24. The gap holds; the headline row is not like-for-like.
The 3.07 with thinking on, the number Nous leads with in prose, is marked provisional, from 2 of 5 runs. It is the row that comes closest to GLM 5.3's 3.10, and it is still below it. The judge is a single commercial model, and the rubric belongs to the lab being judged. No third party has scored this checkpoint.
What is in the download
The repository was created at 19:31 UTC on 6 November, half an hour before the Reddit thread, and last touched at 21:04. It counts 27,356,728,560 parameters in BF16, about 425 million fewer than the base model's 27,781,427,952, a gap consistent with the multi-token-prediction head Nous says it disabled. Qwen shipped Qwen3.8-27B under apache-2.0 as well, so no relicensing argument applies. In roughly its first 36 hours the repo drew 989 downloads and 198 likes.
More usefully, the evaluation is genuinely open: 803 rows in one jsonl, the judge prompt, and both grading scripts. The dataset is not a screenshot of results, it is the apparatus, which means the claim is auditable by anyone willing to spend the tokens. The training pairs are public too. That is the difference between a vendor number and an unreproducible one.
The thread asked the obvious question
The r/LocalLLaMA post drew 259 points and 267 comments, and the top substantive objection is the one the framing invites. "So they replaced eastern propaganda with western propaganda?" wrote Icy-Degree6161, at 207 points. The sharper version came from cinnapear, at 95: "Isn't Qwen already pretty western in its responses? I have never experienced propaganda from it."
The second comment challenges the baseline rather than the fix, and the eval answers it better than the blog does. If base Qwen were already Western-aligned in ordinary use, it would not sit at 22 per cent on these prompts while Inkling and Mistral Large 4 sit at 0.3 per cent. Nous chose prompts from documented statutes and dated reporting, exactly the terrain where a state-aligned model behaves differently from a Western one. Whether that terrain is representative of daily use is a separate question the eval does not ask.
What would settle it
The test that matters is someone outside Nous running the released scripts against the checkpoint, then against a Western model on a conservative-frame prompt set. The eval is Chinese-state-specific by construction; it has no axis for Western official framing, so the symmetry objection the thread raised is not measurable with it. Nous's own thesis, that the values in any open model are only ever one fine-tune away, is testable and much larger than Arete. The Qwen base this model overwrote can be re-tuned by anyone.
The honest summary is narrow. A 27B model that endorsed state framing in 22 per cent of measured answers now does so in under 1 per cent, at the cost of staying mid-pack at stating the facts it was asked about, on the vendor's own test, scored by the vendor's own judge.
