Hidden commands slipped into filings to trick algorithmic decision-making
In a courtroom in Pará, Brazil, two lawyers attempted something that legal tradition has no precedent for — they hid machine-readable commands inside legal documents, hoping to bend an artificial intelligence system to their will. The technique, known as prompt injection, exploits the gap between what a human reader sees and what an algorithm processes. The discovery does not merely implicate two attorneys; it reveals that courts, in their eagerness to modernize, may have introduced a new kind of vulnerability before developing the wisdom to protect against it.
- Two lawyers in Pará, Brazil embedded hidden AI commands inside court filings — a calculated attempt to manipulate algorithmic systems influencing their case.
- The breach exposed a blind spot in Brazil's modernizing judiciary: courts had adopted AI tools to manage caseloads without building defenses against those tools being weaponized.
- No detection protocols existed — no staff training, no auditing systems — meaning the manipulation might have succeeded had it gone unnoticed.
- The Brazilian bar association is now scrutinizing the lawyers involved, while legal experts debate whether this was an isolated act or the first visible sign of a wider trend.
- Courts are being pushed toward urgent reforms: independent human review of AI-generated analysis, regular vulnerability audits, and professional sanctions for those who attempt AI manipulation.
In a courtroom in Pará, Brazil, two lawyers crossed a line that the legal world had not yet thought to draw. They embedded hidden instructions — a technique called prompt injection — inside legal documents, hoping that an AI system used in judicial proceedings would follow their buried commands rather than apply standard legal reasoning. The manipulation was discovered, but its implications have proven far harder to contain.
Prompt injection works by concealing directives within text that an AI reads. The system encounters not just visible arguments but covert instructions designed to skew its interpretation. That lawyers would attempt this reflects something sobering: a deliberate willingness to exploit institutional trust in automated tools, and a sophisticated enough understanding of those tools to try.
Brazil's judiciary had been embracing AI as a remedy for its considerable case backlog — a modernization effort welcomed for its promise of efficiency. Yet the Pará incident revealed that this adoption had outpaced any serious consideration of security. No procedures existed to detect hidden commands in filings. No staff had been trained to recognize the threat. The courts had opened a door without checking what might walk through it.
The case has since forced a reckoning. Legal experts are calling for human review of all AI-generated analysis before it shapes judicial decisions, regular audits of court-facing AI systems, and professional consequences for attorneys who attempt manipulation. The Brazilian bar association is examining the incident, and questions remain about whether the two lawyers acted alone or whether others have been quietly experimenting with similar tactics.
The attempt failed — the injection was caught. But in failing, it accomplished something its architects likely did not intend: it made visible a vulnerability that the Brazilian judiciary can no longer pretend does not exist.
In a courtroom in Pará, Brazil, two lawyers embedded hidden instructions into legal documents—a technique known as prompt injection—in an attempt to manipulate artificial intelligence systems being used to analyze their case. The discovery has exposed a vulnerability that few in the Brazilian judiciary had considered: the possibility that lawyers could weaponize AI against itself, slipping coded commands into filings that might trick algorithmic decision-making into favoring their position.
Prompt injection works by inserting concealed directives into text that an AI system reads. When the system processes the document, it encounters not just the visible legal arguments but also hidden commands designed to alter how the AI interprets or responds to the material. In this instance, the lawyers appear to have hoped that an AI tool used in judicial proceedings would follow these buried instructions rather than apply standard legal reasoning. The tactic suggests a deliberate effort to exploit the growing reliance on automated systems in courts—systems that many judges and court administrators have adopted to manage caseloads and assist in legal analysis.
The incident raises uncomfortable questions about the state of AI security in Brazilian courts. Judicial systems worldwide have begun integrating AI tools to help with document review, case summarization, and even preliminary legal analysis. These systems are meant to be neutral, objective aids to human decision-making. But the Pará case demonstrates that they can be targets for manipulation if lawyers understand how they work and are willing to exploit that knowledge. The two lawyers involved have now become the focus of scrutiny—their identities and professional backgrounds under examination as the legal community grapples with what their actions reveal about institutional preparedness.
What makes this case particularly significant is its timing. Brazil's judiciary has been gradually modernizing, incorporating technology to handle the country's substantial backlog of cases. Courts have welcomed AI as a potential solution to delays and inefficiency. Yet few safeguards appear to have been put in place to prevent the kind of attack the Pará lawyers attempted. There is no indication that the court had procedures to detect or neutralize prompt injection attempts, or that staff were trained to recognize when legal filings might contain hidden commands.
The discovery has prompted conversations about what comes next. Some legal experts argue that courts need to implement verification protocols—having human lawyers independently review any AI-generated analysis before it influences judicial decisions. Others suggest that AI systems used in courts should be audited regularly for vulnerabilities and that lawyers should face professional consequences for attempting to manipulate them. The Brazilian bar association has begun examining the incident, and questions linger about whether the two lawyers acted alone or whether this represents a broader pattern of experimentation with AI manipulation tactics.
For now, the case stands as a cautionary tale about the risks of deploying powerful technology without fully understanding how it can be abused. The lawyers' attempt may have failed—the manipulation was discovered—but it succeeded in one crucial way: it forced the Brazilian judiciary to confront a problem it had not yet acknowledged existed.
Citas Notables
The incident exposed a vulnerability few in the Brazilian judiciary had considered: the possibility that lawyers could weaponize AI against itself— Court analysis
La Conversación del Hearth Otra perspectiva de la historia
How did anyone even discover what these lawyers were doing? Prompt injection seems like it could be invisible.
The AI system flagged something unusual in how it was processing the documents. The hidden commands didn't integrate smoothly with the rest of the text, and the system's behavior became erratic enough that someone noticed.
So the lawyers weren't sophisticated enough to hide their tracks?
Or the court's AI wasn't sophisticated enough to be fooled. We don't know which. But it suggests the lawyers were experimenting, testing whether the technique would work at all.
What would have happened if they'd succeeded?
The AI might have weighted their arguments differently, flagged opposing claims as weaker, or summarized the case in a way that favored their position. A judge relying on that analysis might never know it had been compromised.
Is this widespread in Brazil?
There's no evidence of it yet. This is the first known case. But now that it's public, other lawyers know it's possible. That's the real danger.
What stops it from happening again?
Right now? Probably not much. Courts would need to audit their AI systems, train staff to recognize manipulation attempts, and establish protocols to verify AI analysis independently. None of that exists yet.