AI excels at anything that can be tracked via a benchmark
On its fifteenth anniversary, Sydney creative agency Paper Moose has done something quietly radical: it built artificial intelligence tools not to surrender to the technology's promise, but to sharpen the argument for what machines cannot do. By launching Moose Review and Portal, the agency is staking its future on a philosophical distinction — that creativity possessing genuine spirit, what it calls geist, remains structurally beyond the reach of optimisation and reinforcement learning. In a moment when many creative firms are racing toward automation, Paper Moose is instead mapping the boundary between what should be automated and what must be protected.
- The creative industry is under mounting pressure from AI tools promising faster, cheaper output — and Paper Moose believes most firms are drawing the wrong conclusions from that pressure.
- The agency's core provocation is blunt: AI can master anything with a measurable outcome, but advertising that genuinely moves people requires something no benchmark can capture.
- Moose Review — a synthetic focus group tool validated at 94% accuracy against real human responses — turns that conviction into a practical weapon, letting teams stress-test ideas before spending on production.
- Portal tackles the other side of the equation, automating the scheduling, financial, and operational work that drains agency capacity, so human attention can concentrate where it actually matters.
- The agency's bet is that the winners in this shift won't be those generating the most AI content, but those disciplined enough to know exactly which creative decisions to guard from automation.
Paper Moose, the Sydney creative agency that grew from a filmmaker collective working out of a garage, marked fifteen years in business by doing something counterintuitive: building AI tools designed to defend the limits of AI.
The agency had been quietly developing a thesis. Some work — measurable, optimisable, benchmarkable — could and should be automated. But other work carried what Paper Moose called geist: a spirit or X-factor that made creative output genuinely resonant rather than merely competent. AI systems trained through reinforcement learning were, in the agency's view, structurally incapable of producing it — not because of insufficient data or model size, but as a matter of fundamental design. Chess could be solved because winning was verifiable. Advertising could not be reduced to the same logic.
To act on that conviction, the agency built two products. Moose Review applies decades of marketing science — drawing on researchers like Byron Sharp, Daniel Kahneman, and Karen Nelson-Field — to run advertising concepts through synthetic focus groups before production budgets are committed. The tool claims 94% accuracy against real human subgroup responses and has conducted more than 20,000 reviews since launch. Portal, meanwhile, consolidates the operational machinery of agency life — scheduling, documentation, financial tracking, research — into a single automated system, clearing space for the human judgment that cannot be delegated.
Together, the two products express a single wager about the industry's direction. AI will not eliminate creative agencies, but it will force them leaner and faster, and it will raise the stakes on the work that only humans can do. At fifteen, Paper Moose is arguing that the future belongs not to whoever generates content most efficiently, but to whoever knows precisely which parts of the creative process are worth protecting.
Paper Moose, the Sydney creative agency that started as a filmmaker collective working out of a garage, turned fifteen this year. The milestone arrived with something unexpected: the studio had begun building tools to argue with itself—or more precisely, to argue with the industry's growing faith that artificial intelligence could simply replace human creativity.
The occasion prompted Paper Moose to articulate a thesis it had been developing quietly for some time. The distinction at its core was sharp: certain kinds of work—the kind that could be measured, benchmarked, optimized—could absolutely be automated. Other work could not. The agency borrowed a German word, geist, to describe what it meant: the spirit, the X-factor, the thing that made creative work actually creative rather than merely competent. Current AI systems, trained through reinforcement learning and validated against measurable outcomes, were structurally incapable of producing it. This was not a limitation that better training data or larger models would overcome. It was fundamental.
The agency was blunt about what it saw happening around it. The ecosystem of firms promising fast, cheap AI-generated creative work was, in its view, misguided. "AI excels at anything that can be tracked via a benchmark and optimised through reinforcement learning," Paper Moose stated. "You can train a model to play chess because there is a verifiable outcome: win or lose." Advertising, the agency argued, was not chess. It required something else entirely.
In response to this conviction, Paper Moose built two products. The first was Moose Review, an AI-powered testing tool designed to assess whether advertising actually worked before money was spent on production or media. The framework underlying it drew from decades of marketing science research—the work of Byron Sharp, Les Binet, Peter Field, Karen Nelson-Field, Orlando Wood, Daniel Kahneman, and others who had spent careers understanding what moved human behavior. Moose Review operated by running advertising concepts through synthetic focus groups, posing structured questions and analyzing responses. The results, the agency claimed, mirrored real human subgroups with up to 94 percent accuracy. Since its launch, the tool had conducted more than 20,000 reviews, allowing creative teams to test ideas before committing to expensive production or media buys.
The second product was Portal, an internal software platform that consolidated the operational backbone of agency work—scheduling, documentation, financial tracking, strategic research, media implementation—into a single system. Portal was designed to automate the intelligence-based tasks that consumed agency time and attention, freeing human capacity for the work that actually required judgment, taste, and creative thinking.
Together, the two products reflected Paper Moose's larger conviction about where the industry was headed. AI would not replace creative agencies. Instead, it would force them to evolve. Agencies would need to operate with leaner structures, faster turnaround times, and a sharper, more disciplined focus on the creative work that could not be automated. The competitive advantage would belong not to those who could generate the most content fastest, but to those who could identify and protect the human thinking that machines could not replicate. At fifteen years old, Paper Moose was betting that the future of creativity lay not in fighting automation, but in knowing exactly which parts of the work to automate and which to guard.
Citações Notáveis
AI excels at anything that can be tracked via a benchmark and optimised through reinforcement learning. You can train a model to play chess because there is a verifiable outcome: win or lose.— Paper Moose
A Conversa do Hearth Outra perspectiva sobre a história
When you say AI can't replicate geist, what do you actually mean? Isn't that just another way of saying we don't understand creativity yet?
It's different. Geist isn't mysterious in the sense of being unmeasurable. It's unmeasurable in the sense of being irreducible to benchmarks. A chess move is good or bad. An ad is good or bad based on whether it moves human behavior in a direction that matters to the business. Those are different kinds of problems.
But Moose Review uses AI to predict human response. Doesn't that contradict what you just said?
No. It's using AI to measure something humans do—respond emotionally, behaviorally, intuitively to creative work. The measurement is the AI's job. The creation is the human's job. We're not asking the machine to be creative. We're asking it to be a very precise mirror.
So the real fear isn't that AI will replace creative people. It's that agencies will try to use AI to replace them and fail, producing mediocre work.
Exactly. And they'll do it because it's cheaper and faster in the short term. Portal exists partly to make that temptation less necessary. If you automate the operations work, you don't need to automate the creative work to stay lean.
What happens to the agencies that don't make that distinction?
They become commodity producers. They compete on speed and price. They lose the clients who actually care whether the work moves the needle.