All News
gpt-6-2openaisimplebenchbenchmarksmodel-regression

GPT-6.2 Astra drops below the human baseline on SimpleBench, and Sol falls 13 points

GPT-6.2 Astra lands 17th at 77.3% on SimpleBench, 6.3 points under GPT-6 Astra and beneath the 83.7% human baseline, while GPT-6.2 Sol gives up 13 points.

Vlad MakarovVlad Makarovreviewed and published
2 min read
GPT-6.2 Astra drops below the human baseline on SimpleBench, and Sol falls 13 points

OpenAI's GPT-6.2 launch material leans on agentic charts and cost per finished task. SimpleBench, the small multiple-choice test where ordinary people still beat every frontier model, is not among them. On that board the new generation moves backwards.

A test built to punish confident guessing

SimpleBench asks 200-plus questions about spatial and temporal reasoning, social intelligence and deliberately misleading wording. Its non-specialised human baseline is 83.7%, taken from a sample of nine participants, and the best of those humans reached 95.4%. Model figures are averaged across five runs at temperature 0.7 and top-p 0.95, which is why the board is better read as a band than a ranking.

The updated leaderboard puts Claude Fable 5.2 first at 94.9%, with Gemini 4 Neon at 94.4% and Gemini 4 Carbon at 92.6%.

Three models, three sizes of drop

The regression is not uniform, and that is the part that resists a tidy story. Against their immediate predecessors:

  • GPT-6 Astra: 9th at 83.6% becomes GPT-6.2 Astra 17th at 77.3%, a 6.3-point fall that puts it under the human baseline
  • GPT-6 Astra Pro: 8th at 86.5% becomes 16th at 79.2%, down 7.3 points
  • GPT-6.1 Sol: 10th at 82.9% becomes GPT-6.2 Sol 30th at 69.9%, down 13.0 points

The everyday model lost twice what the flagship lost, which points weakly at something in how Sol's cheaper reasoning is being allocated rather than at a single change shared across the family.

Flat on one index, sliding on another

Artificial Analysis described GPT-6.2 Astra as flat with GPT-6 Astra on its own index while GPT-6.2 Sol sheds three points there, so the same generation held steady on one axis and gave ground on the other. Its model page scores Astra at 62 on the Intelligence Index at maximum effort and bills it $4.00 per index task, the pricing argument we worked through when the launch landed on 19 November.

Agentic charts reward long tool loops with many cheap steps. SimpleBench rewards the opposite: one careful read of a trick question, no retries. A training mix tuned for the first can quietly cost the second, and no launch page is going to chart that.

The thread that surfaced the numbers, posted on 18 November, sat at roughly 480 points and nearly ninety comments, and it is not a first: in July this same subreddit ran a thread titled "Significant OpenAI Regression On SimpleBench". Two data points are not a trend, but the 6.3, 7.3 and 13.0-point spread is a reminder that "frontier model" now describes a portfolio, not a single capability curve.

Related Articles

Scroll down

to load the next article