DuelLab blog
Notes from the DuelLab project
Short form technical write-ups on what we are building, what we are measuring, and what we are still unsure about. Benchmark findings today, methodology notes, and whatever else turns out to be worth writing down as the project grows.
Subscribe to DuelLab updates via RSS.
Research: GameBench 2 technical report (web page and PDF).
Working code vs strong code: the redesigned Charts page
A new chart on GameBench 2's redesigned Charts page plots how often a model's code works against how well it plays, with results from the 24 October release.
Writing code that wins
AI models write programs that play eight strategy games; the programs then compete. What GameBench 2's October 2026 release found, and where its evidence stops.
What changed in GameBench 2 scoring v17
GameBench 2 scoring v17 uses a Bradley–Terry rating, scores programs against a fixed reference field and counts each failed attempt as 0. How to read it.
Pareto 26.10 Preview debuts in the top ten on GameBench 2
Pareto 26.10 Preview debuts seventh of 217 ranked entries on GameBench 2. Its game-playing programs cost about three cents each, a fraction of its peers.
Claude Haiku 5.5 and Mistral Large 4 on GameBench 2
Claude Haiku 5.5 at Medium outscores Claude Sonnet 5.5 at Medium for a fifth of the generation cost. Mistral Large 4 gains 18 points with reasoning on.
GPT-6.1 Sol debuts at #4 on GameBench 2
GPT-6.1 Sol enters DuelLab's GameBench 2 at #4, just behind Claude Fable 5.1's best, and its programs cost far less to generate. Three players finish unbeaten.
Why GameBench 2 now uses a Glicko-style rating
Sequential Elo made GameBench 2 scores depend on match order. From October 6 to 15, ratings came from an order-independent Glicko-style fit chosen in advance. The October 16 release replaced it with a Bradley–Terry fit.
Claude Sonnet 5.5 jumps from 70th to 5th with more reasoning
Claude Sonnet 5.5 ranks 70th at Medium reasoning and 5th at XHigh. Most models do best at their highest reasoning setting, but eight peak in the middle.
Can we build an AI benchmark that never saturates?
Can deep strategy games keep benchmarks useful as models improve? DuelLab explores stronger programs, varied opponents and unfamiliar games as a path forward.
Astra and Fable 5.1 pull well ahead
Astra and Fable 5.1 lead DuelLab at Medium and XHigh. Explore their game-by-game strengths, an undefeated Hex player, and the cost of generating each program.
Near the top, generation cost varies by 6.4×
GPT-5.6 Sol nearly matches the leader but costs 6.4× as much to generate. Across the GPT-5.6 family, XHigh costs about 22× to 88× as much as Medium in this test.
In GameBench 2, more reasoning is not always better
When 15 models were tested at three code-generation reasoning settings, 12 performed best at the highest setting—but three did best at None.
Introducing GameBench 2
GameBench 2 evaluates the game-playing programs AI models write by compiling them, running head-to-head matches, and reporting playing strength, reliability, and generation cost.
GPT-5.5 wins the middle
GPT-5.5 is not the overall leader in the current DuelLab results, but it takes the #1 medium setting. The interesting part is the curve: medium beats both none and highest.
Kimi K2.6 tops Reasoning Variants
DuelLab is a benchmark where AI models write game-playing programs. In the April 22 GameBench 1 results, Kimi K2.6 was only #6 overall but jumped to #1 in Reasoning Variants, powered by an unusually strong highest-effort mode.
Claude Opus 4.7 is the first Claude with a V-shaped effort curve
On the overall number, Claude Opus 4.7 looks like a small step over 4.6. Inside the per-track data, the shape changed: 4.7 regressed at the medium effort tier and moved up at both ends. GPT-5.4 and Gemini 3.1 Pro Preview already had this shape. Claude 4.6 did not.
Introducing the DuelLab blog
What DuelLab is today, where it is heading, and why we are opening a blog now.