Vals AI says Astra beat Factorio: Space Age — with no replay, logs or settings published

Vals AI, the evaluation outfit whose numbers labs quote in launch posts, said on September 17 that OpenAI's GPT-6 Astra has beaten Factorio: Space Age — over 165 hours of in-game time inside two days of wall-clock time — and that the expansion takes a human "about 10-20x as long compared to the standard Factorio." The post collected roughly 2,630 likes and 123 retweets. A screenshot reached r/singularity within the hour, then the game's own r/factorio, where the same image drew on the order of 1,100 upvotes and more than 170 comments. No run log, replay, harness description or cost figure came with it.

The eval company's own standard, and the gap next to it

Vals is not a hype account. Its games benchmark, dated the day after this post, caps each trial at three hours and grades from signed recordings rather than from anything a run reported about itself. The Factorio claim carries none of that. Astra already cleared Portal and RimWorld, covered here, with stream archives and memory files attached. Even the basics are missing: whether Astra drove the shipped game or a scripted interface, and whether the simulation was sped up. The top reply, from Joe, was a joke with roughly 400 likes: "Navier-Stokes: 88 hours / Factorio: 165 hours (I'm aware this is not a legit comparison)."

Two numbers the threads could not reconcile

The arithmetic sits badly. In-game time can only exceed wall-clock time if the game ran faster than real time, and the post never says it did. One commenter, bc_gamez, supplied the mechanism: a person "can only react, click, place their turn/commands only so quickly," so games pause to wait, while an AI "can process and react much quicker." On r/factorio the tone was sharper: "165 hours is nearly 7 days non stop," and "why did Vals AI fail to release the replay and report with this kind of announcement?" Vals also says Space Age has six planets; Wikipedia counts four added to the starting world, and players in both threads asked what the sixth was. The 10-20x human comparison is Vals' own assertion, not a measured study — players put it nearer 2-4x.

What would settle it

Neither Vals nor OpenAI has said whether community blueprints, mods or console commands were allowed, which is what players asked first: "Were blueprints allowed? If so, this isn't that impressive," wrote one reply with 41 likes. On r/factorio the questions were the same — whether Astra was "confined to in-game information only," and, without video, whether it finished. A published replay, the harness and prompt, the world settings and a repeat run would end this in an afternoon. Until then the reading is the one this site reached about the joke FactorioBench thread a week before: a claim is not a result.