Sakana AI's MASS loop lifted one model's score per token 1.2-1.6x, not its intelligence

A Reddit thread titled "Recursive Self-Improvement Is Here" collected 379 points and 61 comments on October 11. What Sakana AI actually published a day later is narrower and more concrete than that headline: a method, MASS, that lets a single open-weights model improve how much score it extracts per token it emits, by running a team of virtual subagents against itself and distilling the result back in. Over two cycles the lab reports a 1.2 to 1.6 times gain in score per output token across four research benchmarks. No absolute capability jump beyond that ratio is claimed, and nobody outside Sakana has reproduced it.

What the ratio measures, and what it does not

The headline number is specific. Over two MASS cycles with Qwen3.6-27B, an open-weights model of 27 billion parameters, the lab says score per output token reached 1.2 to 1.6 times the base model's level across four research benchmarks; the code repository names ScienceAgentBench and MLR-Bench among the public ones. Read that literally and it is an efficiency metric. It divides whatever task score a run earned by the tokens it spent, so a workflow that reaches the same answers with fewer steps raises the ratio without making the model more capable. The lab's supporting comparison is the same kind of claim: a student trained on multi-agent trajectories beats, in its words, a single-agent student trained on 1.4 times more training tokens. That is a statement about how dense the trajectories are, not a capability threshold, and it comes from the same training pipeline that produced the headline number. The tasks are synthetic, open-ended research problems, and no third party has run the loop.

How the loop is wired

One shared base model does all the work, cast as a team of virtual subagents. The system proposes multi-agent workflows, executes them and scores its own output, then runs an evolutionary search over those orchestrations, constrained by structural guardrails that keep the search to shapes it considers legitimate. The best team's executions are distilled back into the single shared model by supervised fine-tuning. The published loop alternates between those two halves, evolutionary workflow optimization and SFT on self-generated trajectories, which is what makes it recursive rather than merely iterative. Sakana sets it out in the announcement and the paper, with code at github.com/SakanaAI/mass. The architecture is the more interesting object here, because it is the part the numbers do not test.

Three findings, one of them inconvenient

Sakana lists three results, and each is a design claim rather than a benchmark row:

• Role generalization: training only on task-solving data from the model being optimized improves the capabilities of the optimizer and the evaluator too, though neither was trained directly.
• Multi-agent trajectories are denser training data than single-agent ones, by the lab's own 1.4x-token comparison.
• Optimizer capacity, not evaluator capacity, can be the bottleneck in how fast the loop improves.

The third is the awkward one. It says the ceiling on improvement speed sits with the part of the system that designs the teams, not the part that judges them, which cuts against the intuition that a strong critic is what an improvement loop needs. It also concentrates the risk, since making the optimizer better is exactly what a self-improving system is supposed to do to itself.

The caveat the lab wrote itself

The authors do not treat the blind-spot problem as boilerplate. "When a model is also its own evaluator, shared blind spots could allow mistakes to be accepted as progress and reinforced through training. For AI to improve beyond human expertise safely, we need to understand these failures and preserve effective ways to detect them and intervene." They state that the community "needs to investigate homogeneous RSI carefully to support AI safety." Homogeneous is the operative word. The loop's evaluator is the same weights as its solver, so whatever the model cannot see about its own output is invisible to the training signal that rewards it, and the reward is what gets distilled.

What the repository concedes

The code release is candid about the gap between paper and implementation. Its notes state that "the current RHI driver uses fixed-reference comparisons and champion arbitration, which differs from Algorithm 1's direct comparison with the retained best output", and that the repository "does not include every paper ablation or analysis", alongside a description of "the current evaluation limitations". That is a difference in the mechanism being run, not a formatting footnote. At the time of writing the repository showed 194 stars, with its latest commit on October 9, three days before the blog post.

The lineage, the reception, and what would settle it

Sakana's Recursive Self-Improvement Lab in Tokyo is not new to this. Its lineage page lists LLM-Squared in 2024, which produced DiscoPOP, the Darwin Godel Machine in 2025, which more than doubled baseline software-engineering performance on SWE-bench, a 30 percentage point absolute gain, ShinkaEvolve in 2025, which solved optimization problems with 150 samples, ALE-Agent, first of 804 human participants in an AtCoder Heuristic Contest, Digital Red Queen in 2026 with MIT, and The AI Scientist, published in Nature in March 2026. Jurgen Schmidhuber joined as Chief Scientific Advisor in September 2026, before this paper.

Reception split by venue. The r/singularity thread drew 379 points and 61 comments; the Hacker News submission of the blog drew 4 points and no comments. Two audiences, two responses, and the quieter one is the forum where the paper would get read.

What would settle the RSI question is not another blog post. It is an independent group running the same loop on a different base model, reporting whether the efficiency gain survives a change of weights, and publishing the failure cases that the paper's own caveat predicts. The blind-spot concern is acknowledged here. Demonstrated is a different thing, and it is the demonstration that would matter.