AI Manipulation Reaches Brazil’s Supreme Court

It’s not unusual for a lawyer to try to influence a judge. But one Brazilian lawyer may have taken that idea a little too far. Beyond making legal arguments, he tried to manipulate the Supreme Court’s artificial intelligence (AI) system.

The tactic is known as “prompt injection.” It involves inserting hidden instructions into a document that an AI system might read and process. The goal is to trick the technology into following those instructions instead of simply analyzing the document.

In this case, the lawyer deliberately buried commands across two pages of a court filing, using tiny white text and fragmented words nearly invisible to human readers but readable by AI. Unfortunately for the lawyer, the trick didn’t work.

The Supreme Court’s technology team detected the hidden instructions. What’s more, Justice Cristiano Zanin’s chambers don’t even use AI to analyze cases or draft judicial decisions.

Nevertheless, Justice Zanin wasn’t amused. He concluded that the lawyer had violated the duty of procedural loyalty under Brazil’s Code of Civil Procedure and imposed a fine equivalent to two minimum wages. He also referred the matter to the Brazilian Bar Association (OAB) for possible disciplinary action and to federal prosecutors to investigate whether a crime had been committed.

As Brazilian courts increasingly adopt AI, lawyers may find new ways to use the technology to their advantage. But there’s a difference between using AI to prepare legal arguments and secretly trying to manipulate the systems that might read them. This lawyer apparently learned that lesson the hard way.