Gemini 4 Carbon tops an index that crowned a different leader the day before
Google handed Gemini 4 Carbon to a subset of cyber defenders and a 24-hour-old #1 ranking. The security scores are vendor-run, and nobody can buy the model.
Vlad Makarovreviewed and published
Google announced Gemini 4 Carbon on October 28, describing it in a post by chief AI architect Koray Kavukcuoglu as "our strongest model with state-of-the-art cybersecurity and long horizon capabilities." The model is not on sale. It goes this week to "a subset of cyber defenders," widening gradually through the Fairwind Program, with no date for developers, businesses or consumers. The three security benchmarks behind the cyber claim are Google's own runs, and the ranking that leads the post belongs to a leaderboard first published the previous morning.
Access is a door, not a ship date
Fairwind is not a waiting list. It is a vetted program: background checks on applicant organizations, phishing-resistant MFA, a ban on reselling access, and permitted work narrowed to defensive tasks such as authorized threat simulation, reverse engineering and malware analysis. The program's own page says it works with more than 650 partners globally, and until this week the model those partners held was Gemini 4 Argon. Carbon is the second consecutive frontier release delivered through that door rather than a product surface, which makes the gating the decision and the launch the footnote.
Ars Technica drew the obvious headline, calling Carbon "its latest frontier AI model that you can't use" and noting the two-week gap since Argon. Google adds that it is "actively engaged in the U.S. government's voluntary process for pre-release model testing," which describes a queue, not a completion certificate.
Three security benchmarks, all of them Google's runs
The cyber case rests on three external benchmarks, presented as the arc of defensive work: find the flaw, prove it is exploitable, then fix the cause.
| Benchmark | Carbon | What the score covers |
|---|---|---|
| SEC-bench Pro | 88.2% | Discovering real vulnerabilities in real software |
| ExploitGym | 54.8% | Turning a known flaw into a working exploit |
| CWE-bench | 71.0% | Patching the root cause rather than the symptom |
Those are vendor numbers, reproduced as Google prints them, and the caveat sits in Google's own sentence about the set: the external benchmarks were "evaluated without guardrails." The scores describe a raw model, not the guarded system a Fairwind partner will operate. The other headline figures are internal, including more than a twofold uplift in writing secure code over Argon and a Chrome team result that Carbon generated "more than 10X" better patches than "another lab's large frontier model," a comparison with no named opponent and no published method.
CWE-bench is the one place third-party data sits next door. DeepMind's Fairwind page carries an older chart, where Argon, Grok 4.7 and GPT-6 Astra tie at 68.0% and Opus 5.5 sits at 67.0%. Carbon's 71.0% is a three-point move at the top of an already crowded chart.
The number one depends on a one-day-old board
Carbon scores 59 on Artificial Analysis Intelligence Index v5.0, the claim Google leads with. The context it omits: v5 published on October 27, and its launch leaderboard had a different winner, Claude Opus 5.5 at max effort on 58, ahead of GPT-6 Astra on 57 and GPT-6.1 Sol on 55, with Gemini 4 Argon next at 50. Artificial Analysis recorded the change on October 28, the same day as Google's post. A one-point lead on a composite that had existed for a day is real and thin.
AA's own write-up is more qualified than Google's citation of it. Carbon leads four of the nine evaluations inside v5 — HLE-Diamond, Terminal-Bench-Science, AA-Omniscience and AutomationBench-AA — and, in AA's words, "is not leading in coding and knowledge work." On the Coding Agent Index it posts 59 with Antigravity CLI against 62 for Claude Code on Opus 5.5, and it is second on Terminal-Bench 4.0 at 64% and FrontierCode v1.1 at 51%, behind the same model.
Cost is where the picture improves: $7.76 per Intelligence Index task, about 23% below Opus 5.5 at $10.04 and roughly 1.2 times GPT-6 Astra at $6.25, on 123,000 output tokens per task.
Where the vendor's own numbers cut both ways
Google's alignment section is unusually specific, and some of it is unflattering to Argon. On Gray Swan's indirect prompt injection benchmark Carbon reports a 0.0% attack success rate, against 0.7% for Argon and, per Ars, 1% for Opus 5.5 and 8.5% for GPT-6 Astra. Successful cheating on Terminal-Bench 4.0 falls to 0.0%, with attempts at 0.4%. High-severity misalignment alerts in internal coding environments dropped by almost 80%, and overreach and under-service by 26% and 29%.
Two things sit awkwardly beside that. Ars observes that Google "positions Carbon as a totally new model, but for all we know, it could be a post-training upgrade of Argon," and that Carbon beats Argon by slim margins on every published test, while the blog says "building on" Argon. Then there is honesty: Google says unsupported claims fell roughly by half, while Artificial Analysis finds AA-Omniscience accuracy up ten points to 58% with the hallucination rate rising from 4% to 11% — still the second lowest of any model, behind Argon, but a near tripling on a benchmark Google did not choose. Both can hold; neither substitutes for the other.
What the announcement leaves out
No parameter count. No training data description, no post-training or reinforcement-learning recipe, no technical report. No context window in the blog, though AA lists one million tokens with text, image, video and speech input and text output. No Carbon-specific safety document: the only one linked is the Gemini 4 report the post attaches to Argon's alignment work.
Pricing is disclosed, and the detail matters. Carbon launches at $2 per million input tokens and $10 per million output, with cached input at 95% off, doubling to $4 and $20 when the introductory period ends. Ars reports Google declined to give an end date when asked, unlike Argon, whose discount expires at the end of November. And no third party has published its own run of the three: the leaderboards carry the numbers, but the measurement is Google's. AA's index, the one external scoreboard in the post, is a general capability composite and does not run SEC-bench Pro, ExploitGym or CWE-bench.
What would settle it
The cyber claims need a reproduction by someone with no stake in the result, on the published harnesses, with per-task outcomes, not a single percentage. Fairwind supplies the second test: fix-rate telemetry from 650 partners — how many findings were real, how many patches survived review, what the false positive rate looked like at volume — is evidence a benchmark cannot supply, and none of it is public. A general availability date and a Carbon system card would answer the rest.
The accurate summary is therefore narrow. Google has what appears to be the strongest general-purpose system on a one-day-old composite, a real improvement in a domain where it had been behind, and an access policy that keeps every operational claim unverifiable by design. The ranking is real. The security record is a press release.
