Showing posts with label phishing. Show all posts
Showing posts with label phishing. Show all posts

Tuesday, September 1, 2026

Sieve or Shield? : Why AI Text Watermarks are Remarkably Easy to Scrub

 

In a nutshell (TL;DR)...

While AI text watermarks such as Google DeepMind's SynthID-Text are legally mandated by the EU AI Act to identify synthetic content, independent security research demonstrates that they are technically fragile and easily scrubbed using inexpensive paraphrasing tools. This vulnerability highlights a major divide between regulatory ambitions and technical reality, proving that existing watermarking techniques cannot serve as a reliable defense for content verification.


When the European Union drafted the transparency rules for the EU AI Act, regulators envisioned a digital ecosystem where artificially generated text would be permanently and reliably stamped. Under Article 50(2), AI providers are legally required to mark their synthetic outputs in a machine-readable format so that downstream detection tools can identify AI-generated content.

To comply, companies like Anthropic have deployed advanced statistical watermarking frameworks like Google DeepMind’s SynthID-Text globally. But while these watermarks have sparked outrage over potential drops in writing quality, independent security research has revealed an even deeper, structural flaw: for motivated bad actors, AI text watermarks are incredibly easy to erase.

The Illusion of "Robust" Marks

On paper, statistical text watermarking is designed to be highly resilient. Because the watermark is embedded directly into the statistical patterns of word choice, rather than as a hidden character or metadata tag, it travels with the text when copied, pasted, or slightly modified. Anthropic and DeepMind note that the watermark can survive mild paraphrasing, minor word substitutions, and cropping.

However, "surviving mild editing" is a far cry from being cryptographically secure.

Independent researchers at the SRI Lab at ETH Zurich conducted a comprehensive, adversarial audit of the open-source SynthID-Text framework. Their findings were stark: while SynthID-Text is highly resistant to "spoofing", meaning it is very difficult for an attacker to forge Claude’s watermark and falsely attribute human-written text to the AI, it is remarkably fragile when it comes to "scrubbing" (completely removing the watermark).

According to the SRI Lab evaluation, even naive adversaries using off-the-shelf, baseline paraphrasing tools can easily bypass or strip SynthID-Text watermarks. In their tests, standard rewriting tools successfully "scrubbed" the watermarks from text, rendering them completely undetectable to the watermark classifiers.

The $50 Attack: How Watermarks are Bypassed

Why is a watermark so easy to wash away? The vulnerability lies in the very nature of language.

When an AI watermarking algorithm like SynthID-Text generates text, it is essentially applying a slight mathematical bias (using its secret g-function) to favor certain words over others. But as soon as that text is fed into a secondary AI model, such as a paraphrasing tool or a translation engine, that secondary model completely replaces those biased word choices with its own, unbiased vocabulary distributions.

The SRI Lab research demonstrated that an attacker can use a technique called a "stealing attack". By sending a series of black-box queries to the watermarked LLM, the attacker can learn the statistical pattern of the watermark. Once the attacker understands the boundary of the watermark, they can apply "assisted scrubbing".

The results are devastating for the watermark’s credibility: the scrubbing success rate soared to above 90%, and in some cases reached nearly 100%. The financial barrier to executing these attacks? Academic researchers noted that a successful attack could be carried out for under $50.

Furthermore, the researchers found that DeepMind's use of "tournament sampling" actually made the watermark more sensitive to rewrites and easier to scrub than more basic watermarking schemes, as the mathematical g-values are highly fragile when sentences are reorganized.

The Rise of GitHub Bypasses

This academic vulnerability is already playing out in the real world. ZDNET recently reported a massive public backlash against watermarking, which has triggered a sudden surge of "watermark-removal" and "re-humanizing" tools on GitHub.

One notable tool, Declaude, was specifically designed to strip the statistical markings left by Claude’s models. James Padolsey, the developer behind Declaude, criticized the underlying EU watermarking mandate as an arbitrary, "feel-good" regulation. Padolsey pointed out that the policy mostly penalizes ordinary, law-abiding users who get flagged for using AI for benign tasks like proofreading, while doing virtually nothing to stop deliberate bad actors from using simple scripts to scrub watermarked text before deploying it in misinformation or phishing campaigns.

The Regulatory Disconnect

The ease of scrubbing reveals a deep chasm between European regulatory ambitions and computer science realities. Under the EU AI Act, regulators are developing a voluntary Code of Practice on Transparency that demands "robust" and "reliable" watermarking technologies.

Yet, as DeepMind itself admits, a watermark’s confidence score is heavily degraded the moment a text is thoroughly rewritten, translated into another language, or mixed with human-written text.

For businesses and compliance officers, this technical reality means they cannot treat watermarking as a silver bullet for content verification. If a company's compliance strategy relies solely on detecting watermarks to prevent AI-generated misinformation from slipping through its pipeline, its defenses are effectively a sieve.

In Part 3 of this series, we will step out of the technical sandbox and look at the legal and financial hammer that the European Union is preparing to swing at companies that fail to master these transparency rules.


The August Deadline Most Boards Missed : Inside the EU AI Act’s Article 50

  In a nutshell (TL;DR)... Active Deadline: Article 50 transparency obligations became active on August 2, 2026. Scope: Applies to any AI sy...