All News
eu-sovereign-aieuropajupitermixture-of-expertspytorchtorchtitaneurohpcsupercomputers

EUROPA kept a 951.9B-parameter model alive on a shared supercomputer. What it can do is still unmeasured

EUROPA took a 951.9B-parameter MoE from 90 to 1,293 tokens/s per GPU on shared public hardware. The engineering is documented. The model's ability is not.

Vlad MakarovVlad Makarovreviewed and published
5 min read
EUROPA kept a 951.9B-parameter model alive on a shared supercomputer. What it can do is still unmeasured

On 12 November the EUROPA consortium published a long engineering report about pretraining a mixture-of-experts model on JUPITER, the EuroHPC exascale machine at Forschungszentrum Jülich. The headline number is 951.9 billion parameters. The number that carries the actual result is 1,293 tokens per second per GPU, up from 90 at the start. Getting from one figure to the other took a set of fixes to PyTorch and TorchTitan that the consortium has now upstreamed, and those are the part other labs can use. The model is not. It has no benchmark, no evaluation, no weights and no release date.

Running a trillion-parameter model is a hardware purchase. Making one train at usable speed on a machine designed for simulation, shared with hundreds of projects and wired with links 36 times slower than the one inside a single superchip, is an engineering problem, and the full report documents it properly. Whether the model is any good is unknown, and nobody outside Domyn, Fraunhofer IAIS and Fraunhofer IIS can check.

The mandate was 400 billion, not a trillion

The EuroHPC Frontier AI Grand Challenge was built to test whether sovereign, open AI could reach frontier scale on public money. EUROPA, a consortium of Domyn with Fraunhofer IAIS and IIS, won it in June 2026 with a proposal for an open European MoE of at least 400 billion parameters, support for all 24 official EU languages, strict EU AI Act compliance and access to 2.5% of EuroHPC capacity. The run described here is roughly double that floor, and it is a configuration used for experiments: some values, the companion post notes, may change before the training run starts.

PropertyValue
Total parameters951.9B
Active per token32.8B
Layers / hidden size74 / 6,144
Experts384 routed, top-8
Expert hidden size1,792
Vocabulary256,000
Pretrain sequence65,536 tokens
OptimizerMuon + AdamW

Three fixes, measured one at a time

The report walks a single curve, 90 to 1,293 tokens per second per GPU, and names the change behind each step. Three account for the biggest jumps.

ChangeBeforeAfter
FlexShard Muon placement, Newton-Schulz on GPU441 tok/s, step 41.7 s597 tok/s, step 2.2 s
Activation-checkpoint offload to Grace370 tok/s508 tok/s
FSDP all-gather on compute stream996 tok/s, 88.1 GiB peak1,210 tok/s, 74.6 GiB

The Muon problem was geometric. Muon orthogonalises each update across a whole matrix, but FSDP had split every matrix 384 ways, so the naive version gathered each one onto a single rank and scattered it back. FlexShard, now part of TorchTitan, let the team shard experts by whole expert, so each rank owns complete matrices and runs Newton-Schulz with no communication. Moving that work from Grace's 72 Arm cores to the Hopper GPU, with parameters still offloaded, cut the optimizer step by a factor of nineteen.

The second was arithmetic. At 65,536 tokens the boundary activations alone come to 74 layers times 65,536 times 6,144 times 2 bytes, about 55 GiB per GPU, which does not fit in 96 GB of HBM beside weights and gradients. Copying them to Grace over the 900 GB/s NVLink-C2C link and pulling each back a layer before it was needed bought sequence length first, then throughput.

The third may outlive the project. After a software upgrade, every rank began retrying allocations at around 70% memory utilisation with 25 to 30 GiB apparently free. FSDP's side streams were holding 27.3 GiB, of which 6.4 GiB was live. Running the all-gather setup on the compute stream, via a private flag called unshard_async_op, took ranks with retries from 384 to zero. That flag is being promoted into a public FSDP API, so the benefit is not EUROPA's alone.

The bottleneck was the wire, not the FLOPS

Each JUPITER node holds four GH200 superchips: a 96 GB HBM3 Hopper GPU paired with a 72-core Grace CPU over a 900 GB/s link, GPUs joined at 150 GB/s per direction, and four InfiniBand adapters giving about 25 GB/s per GPU to the rest of the cluster. Compute nodes carry the 120 GB Grace configuration rather than the 480 GB on login nodes, which leaves about 108 GiB of host memory per GPU after kernel overhead, and sometimes less.

That ratio set the layout: 384-way FSDP, 4-way expert parallelism confined inside each node so the MoE token exchange stays on NVLink, and no pipeline parallelism. A batch of 384 sequences of 65,536 tokens puts roughly 25 million tokens in a step, and the final configuration runs 1,536 GPUs across four data-parallel replicas. The companion post's own table puts per-GPU throughput at four replicas at 89.4% of the single-replica figure while calling the effect inappreciable, which is worth noting before quoting 1,300 tokens per second as a flat rate.

Reading "one trillion" correctly

951.9 billion is a parameter count, and 32.8 billion of those parameters do work on any given token. That is a large model and not the largest. Mistral's Mistral Large 4, shipped as Le Chonk, is a trillion-parameter MoE with 52 billion active, trained on 3,800 Grace Blackwell GPUs in the company's own datacenters, and its weights landed on 29 October. EUROPA's run is on older Hopper silicon, on a machine it does not own, with 32.8 billion active parameters against 52 billion for Le Chonk. Different achievements, and only one comes with a downloadable artifact.

The word open needs the same care. The consortium says the model and its training recipes will be released when the EUROPA model is published. That is a commitment, not a file. The 400B figure is the floor the commission bought; the 951.9B configuration is a feasibility experiment above it. Which of the two ships, at what quality and under which licence, is open.

What the thread got right

Reaction on r/eutech, where a thread on the run drew around 190 points and 52 comments, split along a predictable line. The top objection was that the achievement is smaller than the framing: "Running a 1T model isn't impressive, training is slightly better but still not an achievement for someone big as EU." Others reached for scale, citing Kimi K3 at 2.8 trillion parameters and noting that Mistral already has a trillion-parameter model. A quieter counter-argument was that public infrastructure is the binding constraint and this is a signal, not a catch-up.

On capability the skeptics are ahead by default, because there is nothing to measure. On infrastructure the report makes a case the parameter count obscures: the claim is not that Europe can build a big model, it is that a machine designed for fluid dynamics and shared across hundreds of projects can hold one at all.

What would make this checkable

One number would move the story from engineering to AI, and it is an evaluation. The report lists what its authors have not resolved, including how severely group-limited routing costs model quality over a long run and whether the expert GEMMs survive FP8 without measurable loss. Those are honest admissions from people who know where the soft spots are, and they are also what decides whether 951.9B parameters become a model worth running.

Until then, read the post as a detailed, reproducible recipe for training very large sparse models on hardware you do not own and cannot reconfigure, with measured gains at each step and merged pull requests to prove it. That is a real contribution to European capacity. It is not evidence about model quality, and the consortium has not claimed otherwise. The gap is in the headlines, not the report.

Related Articles

Scroll down

to load the next article