FAQ and glossary
Reading DuelLab GameBench 2
Is a score a percentage of games solved?
No. A game score is the share of points a player program is expected to win against a fixed reference field, on a scale from 0 to 100 (a win is worth one point, a draw half). The field holds every player program that was rated when the field was set, with equal total weight for each model, and stays the same until DuelLab announces a new version. A model’s score averages its games; within a game, each attempt whose model-written code failed counts as 0 beside the scores of its working programs. Every release re-estimates program strengths from all matches, so scores can still change when match evidence, models, or games are added.
Does GameBench 2 use human or AI judges?
No human or AI judge grades how intelligent a player looks or decides which program deserves the higher score. Programs compete in deterministic games, and match outcomes produce the ratings. People still define the benchmark, select games, and review its methods.
What does DuelLab count?
The summary separates model families, model-and-reasoning-setting rows, Overall models, player programs, and games. It also separates complete from partial Overall results and ranked from provisional Reasoning Variants results.
What is the Reasoning Variants leaderboard?
It shows each tested model and reasoning setting as a separate row. Provider settings are not a standard amount of computation, so “High” for one model should not be treated as identical to “High” for another.
How does Overall ranking work?
A model’s Overall score averages its Baseline, Balanced, and Intensive scores. Since October 13, 2026, the Overall leaderboard ranks only models with scores for all three reasoning settings, documented as three distinct provider settings. An average of fewer settings leaves out the missing ones and is not comparable, so those models are listed below it as Provisional, without an Overall rank; each of their settings can still be ranked in Reasoning Variants. A setting in which every attempt failed in the model’s own code scores 0 and still counts as one of the three.
How can a model be Partial 3/3?
Three of three means all three setting scores are present and the settings are documented as distinct. A Partial 3/3 model is officially ranked, but another evidence or quality check is still missing, such as enough working player programs or enough match coverage in one setting. It keeps its numeric rank and shows the reason beside it.
What is a player program?
It is code written by a model to play one public game. Some downloadable files call it an “entrant”; in those files, the terms mean the same program.
If the games are repeatable, why run several matches?
Matches can change the starting side, opening position, game setup, opponent, or player program. Running these different situations gives a fairer picture than repeating one identical match.
What does Provisional mean?
The row does not have an official rank. In Reasoning Variants, a rank needs an evaluated attempt in all eight games (one that produced a program or failed in the model’s own code), enough working player programs, and enough match coverage. The row remains visible so readers can see the result collected so far. In a few rows, a game can never be evaluated: a limit DuelLab set cut the model’s reply short, and the original request can no longer be repeated. Such a row’s score averages its other games, and the row stays unranked. In Overall, Provisional means the model lacks a score for one of the three settings or three distinct documented settings.
What do the extra columns mean?
Working-program strength averages the scores of a model’s working programs across games, leaving failed generation attempts out. Codegen shows whether programs worked on the first try, after repair, or not at all. The Players column counts the programs behind a result. Best is the row’s highest single-game score. Reasoning spread compares a model’s tested reasoning settings. W / D / L shows rated wins, draws, and losses. Opponents played, an optional column, shows how far the row’s programs are through the full schedule of matches.
Why can a filtered rank differ from the official rank?
A filter ranks only the rows or game currently shown. DuelLab keeps the official full-leaderboard rank visible so the temporary filtered order is not mistaken for the published rank.
How should I read estimated cost?
Std. price is an equal-weight standard-price equivalent for generating and repairing one player program. It can use billed, provider-reported, or versioned price evidence, so it may differ from an invoice. $0.00 means exactly zero; a positive amount below one cent still appears as a positive value.