The AI becomes not a neutral arbiter but another tool that favors those with knowledge to exploit it
In Brazil's courtrooms, a quiet reckoning is underway as legal professionals have discovered how to bend the artificial intelligence systems now woven into judicial proceedings — not through brute force, but through the subtle grammar of algorithmic confusion. The Superior Court of Justice has opened investigations and moved toward disciplinary action, confronting a paradox as old as law itself: that any system designed to deliver fairness can become, in the wrong hands, an instrument of advantage. The crisis asks a question that transcends technology — whether institutions can modernize faster than the human impulse to find the seam in every new structure.
- Brazilian lawyers have been quietly exploiting AI vulnerabilities in court systems — embedding misleading language, unusual formatting, and strategic keywords to distort how algorithms process their filings.
- The manipulation has grown from isolated incidents into what investigators now recognize as a systemic pattern, with informal networks of attorneys apparently sharing techniques with one another.
- The Superior Court of Justice has launched formal investigations and pursued disciplinary action against multiple attorneys, though it has kept names and specific methods out of the public record.
- Courts are scrambling to respond — adding human review layers, working with software vendors on patches — but the underlying logic of the exploit remains difficult to fully close.
- The crisis threatens the foundational promise of judicial AI: that automation would make the system faster and fairer, not simply create a new arena where the most technically sophisticated gain the upper hand.
Across Brazil's courtrooms, a quiet crisis has taken shape. Lawyers have begun exploiting vulnerabilities in the AI systems that now sort filings, summarize cases, and assist judicial decisions — not crudely, but with surgical precision. A strategically confusing document, unusual formatting, or carefully repeated keywords can cause an algorithm to misweight a case, elevate an undeserving motion, or bury a deadline. Courts dependent on these tools to manage overwhelming caseloads were slow to recognize what was happening.
The Superior Court of Justice has since taken notice. What began as scattered reports from judges and opposing counsel has revealed itself as something more organized — a growing cohort of legal professionals who understand how to game the machines meant to make justice more consistent. The STJ has already moved to discipline multiple attorneys, though it has not named them publicly or detailed the specific techniques involved. Institutional caution may explain the discretion: publicizing the exploits risks spreading them further. Proving intent is also difficult — a lawyer can always claim that odd formatting is simply personal style.
The problem cuts to the heart of Brazil's judicial modernization project. Courts embraced AI as a remedy for chronic backlogs and inconsistency, trusting that algorithmic processing would be more neutral and reliable than overburdened human review. But a system that can be systematically deceived is no longer neutral — it becomes another lever that favors those with the knowledge and resources to pull it. Some courts have responded by adding human checkpoints for AI-flagged cases; others are working with vendors to patch the weaknesses. Neither solution fully resolves the underlying tension.
The investigation continues, and the STJ's signal that it will not tolerate the practice appears to be reaching the legal community. But the deeper lesson may be institutional: modernization does not eliminate vulnerability, it relocates it. And the lawyers who practice before these systems have so far proven faster at finding the new seams than the courts are at closing them.
Across Brazil's courtrooms, a quiet crisis is unfolding. Lawyers have begun systematically exploiting vulnerabilities in the artificial intelligence systems that now process legal filings, generate case summaries, and assist judges in their decisions. The manipulation is not crude—it is surgical. A lawyer submits a document designed to confuse the AI, embedding language that the system misinterprets or misweights. The algorithm flags the case differently than it should. A motion gets priority it doesn't deserve. A deadline gets lost in the noise. The courts, increasingly dependent on these systems to manage their crushing caseloads, have been slow to notice.
But notice they have. The Superior Court of Justice, Brazil's second-highest tribunal, has begun investigating the practice. What started as isolated incidents reported by judges and opposing counsel has revealed itself to be something more systematic—a growing number of legal professionals who understand how to game the machines that are supposed to make the system fairer and faster. The STJ's investigation has already led to disciplinary action against multiple attorneys, signaling that the institution takes the threat seriously.
The scope of the problem remains unclear. The STJ has not released comprehensive data on how many cases may have been affected or how many lawyers are involved. But the fact that the court felt compelled to act suggests the issue has moved beyond anecdote. Judges report encountering documents that appear designed specifically to trigger algorithmic errors—filings with unusual formatting, strategic repetition of keywords, or arguments structured in ways that seem calculated to confuse rather than persuade. Some lawyers have apparently shared techniques with colleagues, creating informal networks of practitioners who understand the exploit.
The vulnerability exposes a deeper tension in Brazil's judicial modernization. The country's courts have embraced AI systems as a solution to chronic backlogs and inefficiency. These tools promise to sort cases faster, identify relevant precedents more reliably, and reduce the subjective judgment that can lead to inconsistent outcomes. But the systems are only as reliable as their training data and their resistance to manipulation. If lawyers can systematically deceive them, the entire premise collapses—the AI becomes not a neutral arbiter but another tool that favors those with the knowledge and resources to exploit it.
The disciplinary cases against the attorneys involved have been relatively quiet affairs, handled through professional conduct channels rather than criminal prosecution. The STJ has not named the lawyers publicly or detailed the specific techniques they used. This discretion may reflect institutional caution—the court may not want to publicize the vulnerabilities, fearing it would encourage wider adoption of the exploits. Or it may simply reflect the reality that proving intent is difficult. A lawyer can argue that unusual formatting or repetitive language is simply their writing style, not a deliberate attempt to confuse an algorithm.
What happens next will likely depend on how quickly the STJ and the technology companies behind these systems can respond. Some courts have begun implementing additional human review of AI-flagged cases, essentially creating a second checkpoint. Others are working with their software vendors to patch the vulnerabilities. But the underlying problem—that AI systems can be gamed by those who understand their logic—is not easily solved. It is a problem that will persist as long as the stakes remain high and the incentives to cheat remain strong.
For now, the investigation continues. The STJ has signaled that it will not tolerate the practice, and that message appears to be reaching the legal community. But whether that is enough to stop the spread, or merely to slow it, remains to be seen. The courts are learning, in real time, that modernization brings its own vulnerabilities—and that the lawyers who practice before them are often faster at finding them than the institutions are at closing them.
Citas Notables
The STJ has signaled that it will not tolerate the practice, and that message appears to be reaching the legal community— Court response to the manipulation scheme
La Conversación del Hearth Otra perspectiva de la historia
Why would lawyers risk their careers to manipulate an AI system? What's the actual payoff?
The payoff is the same as it's always been—winning cases, getting favorable rulings, protecting clients. But the AI systems are opaque enough that a lawyer might think they can exploit them without getting caught. The system flags something as low priority, a motion gets delayed, a deadline slips. Who knows if it was the algorithm or just the court's normal chaos?
But the STJ found them. So the systems must leave traces.
They do, but only if someone is looking for them. A judge has to notice that a filing is oddly formatted, or that certain keywords appear too many times. Most judges are drowning in cases. They might not see the pattern until it's too late.
So this is about the courts being understaffed and overwhelmed.
Partly. But it's also about the fact that we've handed decision-making power to machines that nobody fully understands. Even the people who built the AI can't always explain why it made a particular choice. That opacity is an invitation to manipulation.
Is this unique to Brazil, or is this happening everywhere?
Brazil is reporting it, which is what matters. Other countries probably have the same vulnerabilities. But Brazil's courts are particularly dependent on AI to manage their backlogs, so the incentive to exploit the systems is higher. And the legal culture there may be more willing to push boundaries.
What does the STJ actually do now? Can they fix this?
They're adding human review, patching vulnerabilities, trying to make the systems harder to game. But you can't eliminate the problem entirely. As long as there's a machine making decisions, someone will try to trick it. The question is whether the courts can stay ahead of the tricks.