A research firm has put a new yardstick on the table, one meant to measure whether artificial agents can handle work that carries real economic weight rather than parlor puzzles. Its first public reading is a humbling one: the strongest model it tested, Opus 5.5, completes only about twelve percent of the tasks. Whether that figure marks a wall or merely a starting line depends on details not yet visible, but it reminds us how far the ambition of automated labor still runs ahead of its present ability.
SemiAnalysis Unveils GDPx Benchmark: Best Frontier AI Model Scores Just 12% on Valuable-Domain Agent Tasks
A new yardstick exists, and its first reading is 12%.
Mark
So what actually happened here? A benchmark launched, and the top score is 12%?
Mimi
That's the core of it. SemiAnalysis released GDPx, which tests frontier models as agents across economically valuable domains, and Opus 5.5 leads at 12%.
Mark
Why does the word agent matter?
Mimi
It suggests the models are being asked to carry out tasks, not just respond to prompts. That's closer to how people imagine AI doing real work.
Mark
And 12% sounds low. Does it mean AI is mostly failing at valuable work?
Luke
Careful. We don't know what the 12% measures. It could be fully completed tasks, partial credit, or a weighted score. Without that, the number can't carry a claim like that.
Mimi
Fair. The summary says agents complete only a small share of these tasks, but that's an interpretation of the number, not a detail from the methodology.
Mark
What else is missing?
Luke
How many tasks, which domains, which other models were tested, and how they scored. Even what GDPx stands for isn't in the material. Also, the 12% comes from SemiAnalysis's own benchmark, so it's one source's measure.
Mark
Then what's worth watching?
Mimi
Whether later model releases push the score up, and whether others adopt GDPx as a reference point.
Luke
And whether the methodology holds up once people read it closely.
El Pulso
- A new benchmark called GDPx has surfaced, testing AI agents on tasks drawn from the most economically valuable domains.
- The headline number is stark: the leading model, Opus 5.5, scores just 12 percent, and the figure is spreading across business and crypto outlets alike.
- The full methodology remains unclear, leaving open what a 12 percent score actually measures and how difficult the tasks were designed to be.
- The trajectory now rests on the next wave of model releases, which will show whether scores climb meaningfully or stall near their current low point.
A research firm has put a new yardstick on the table, one meant to measure whether artificial agents can handle work that carries real economic weight rather than parlor puzzles. Its first public reading is a humbling one: the strongest model it tested, Opus 5.5, completes only about twelve percent of the tasks. Whether that figure marks a wall or merely a starting line depends on details not yet visible, but it reminds us how far the ambition of automated labor still runs ahead of its present ability.
Three outlets carried nearly the same news within a short span: SemiAnalysis has released a benchmark called GDPx, and the best model it tested managed a score of 12 percent. The benchmark is described as an agent test, meaning it asks models to act rather than merely answer, and its tasks are aimed at high-value economic domains where commercial stakes are real.
The number is the part that travels. Read plainly, the top model completes only a small share of the work, suggesting that current agents remain far from handling most of this kind of task. Yet the material does not reveal what a score of 12 percent counts, whether fully finished tasks, partial credit, or some weighted measure, nor how other models fared or how the domains were chosen.
So the story is a thin one, and it is best held that way. A new yardstick exists, and its first result is sobering. What follows will tell us whether GDPx scores rise in later model releases and whether others adopt it as a serious measure of economic AI capability. Until the methodology is read in full, the 12 percent is one firm's first reading, not a final verdict.
The headline arrived as three lines stacked on top of one another, each from a different outlet, each saying nearly the same thing: SemiAnalysis has released a new benchmark called GDPx, and the best model it tested scored 12%.
That is close to the full extent of what the available material establishes. The benchmark is named GDPx. It comes from SemiAnalysis. Its stated purpose, according to the title of the firm's own piece, is measuring frontier model capabilities across the most valuable domains. One of the headlines identifies the leader as Opus 5.5, and says it tops what is described as SemiAnalysis's GDPx Agent Benchmark at 12%.
The framing matters. GDPx is described as an agent benchmark, which points to tasks where a model acts rather than simply answers. It is also aimed at high-value economic domains, which suggests the test is meant to reflect work with real commercial weight, not trivia or abstract puzzles. The name itself hints at that economic orientation, though the material does not explain what the letters stand for or how the domains were chosen.
The 12% figure is the number people will carry away. Read plainly, the top-scoring model completes only a small share of the tasks in the benchmark. The editorial summary draws the same conclusion: current agents are still far from handling most of this work. The material does not say what a score of 12% actually counts, whether fully completed tasks, partial credit, or some weighted measure.
Three outlets carried the news in the snippets available: the SemiAnalysis release itself, FourWeekMBA, and Crypto Briefing. The pairing of a technical research firm with business and crypto-oriented aggregators shows how widely the figure is traveling. The detailed methodology, the number of tasks, the list of other models tested, and their scores are not included here.
So the story is a thin one, and it is better told that way than padded. A new yardstick exists. Its first public result puts the leading model at 12%. Whether that is a sobering number or simply the starting point of a hard test depends on details that were not in the material: how difficult the tasks were designed to be, and how other models fared.
What to watch is straightforward. The next round of model releases will show whether GDPx scores move meaningfully upward from 12%, and whether other labs and analysts begin citing the benchmark as a measure of economic capability. Until the full methodology is read, the figure is best treated as one firm's first reading, not a settled verdict.
Quieres la nota completa?
Lee el original en Google News ↗
Technology