Anthropic’s LLM watermarking
So yeah, Anthropic has announced that it’s now watermarking the outputs of Claude, using a scheme based on Google’s SynthID, which is in turn based on the Gumbel Softmax scheme that I proposed at OpenAI back in 2022—as far as I know, the first LLM watermarking proposal, though far from the last one. I’m gratified that Anthropic credits me for this, even though I shirked my duty by never publishing a paper about it (by the time I sat down to write one, it seemed like the whole field had already assimilated my scheme and moved beyond it—AI just moves too fast for me!).
For those who don’t know, watermarking means slightly changing the way that an LLM operates to insert a subtle signal that lets you prove later, with high statistical confidence, that a text indeed came from your specific LLM. It uses the randomness that’s already present anyway in LLM outputs, replacing some of it by pseudorandomness that favors certain word combinations over others in a way that’s later detectable, given only the sequence of tokens itself (not the prompt or the probabilities) along with the key of the pseudorandom generator. Christ, Gunn, and Zamir then substantially improved my scheme to get true cryptographic indistinguishability, and there have been other improvements since.
I’d been meaning to blog about this for days. Thankfully, Zvi Mowshowitz, the world’s foremost blogger about AI, has now written a wonderful post, entitled AI Text Watermarking Is Free And Good, which saves me from the need to write my own long post. In particular, Zvi masterfully explains the central point that I needed to explain to everyone back in 2022-23: why, contrary to many people’s intuitions, there’s no inherent tradeoff between watermarking and the quality of LLM output. Basically, nearly every LLM output was already a sample from a cloud of exponentially many possibilities, all of them about equally good, so there’s plenty of room to steer within that cloud without affecting anything that an ordinary user would notice. As my kids would put it, the math mathes.
As Zvi explains, the central technical drawback of watermarking schemes like the one I proposed, and what Anthropic is now using, is that it’s possible to remove the watermarks with a little extra work (even stuff as simple as, e.g., translating between English and French, asking the LLM for words interspersed with emojis and then removing the emojis, or using an open model to paraphrase the output). Zvi gives detailed arguments for why he expects watermarking to remain a net positive in practice despite this vulnerability.
I could add that, in addition, there’s recent progress (see here for example) on what I’ve called “semantic watermarking,” or watermarking at the level of the underlying concept vectors rather than the tokens themselves. This actually seems to work, albeit with no theoretical guarantees, and will hopefully make removing watermarks a lot harder—although the Barak et al. impossibility result suggests that under plausible assumptions, no LLM watermarking method will be completely foolproof.
Anyway, I worked out my scheme in Fall 2022, then gave lots of talks about it (including, as it happens, at Anthropic), and also worked with Hendrik Kirchner at OpenAI, who actually implemented and tested my scheme. Unfortunately, OpenAI leadership decided against deploying watermarking, worried mostly about risks to the product (i.e., customers disliking the idea, and leaving for a competing LLM that doesn’t watermark). You can read this Wall Street Journal investigation from two years ago for more. I was hopeful that the State of California was going to solve the collective-action problem by mandating watermarking for AI models, but then they decided to do that for audiovisual content only, for some reason exempting text.
Nevertheless, Google DeepMind implemented something very similar to my proposal in its SynthID, deployed in all its Gemini text models. But they heavily restricted who gets to detect the watermark, which made their admirable decision of limited use to my academic colleagues, who’ve been begging me for a way to detect whether their students are using AI to cheat. (For now, I mainly send them to Pangram, a leading AI detector not based on watermarking, as a first line of defense.)
And now, apparently to comply with EU regulations, Anthropic says they’ve deployed a watermarking scheme like mine where anyone will be able to do detection (though they also say in their FAQ that they’re still working on the detection API). Even OpenAI suggests that it plans to follow suit. So, four years after I seriously thought about this, it looks to my surprise like this is actually happening. Thanks, EU!
Tell you what: read Zvi’s post, and then whatever questions you still have, you can come here and ask in the comments. Just please don’t use Claude to write the comments. With any luck, I’ll eventually be able catch you if you do.
Follow
Comment #1 August 22nd, 2026 at 5:43 pm
Over at Daring Fireball, Gruber (a pretty clever tech guy I presume you’ve heard of) was quite harshly critical of this approach:
https://daringfireball.net/2026/08/anthropics_watermark_text_adulteration_in_claude_is_a_perversion_of_writing
Care to comment?
Comment #2 August 22nd, 2026 at 6:08 pm
Joe #1: No, I hadn’t heard of him, and he seems like an ignoramus. He fundamentally doesn’t understand how making pseudorandom choices is not detectably different to the end user from making truly random choices—indeed, that’s the entire point of pseudorandomness.
As I said, there are cogent objections to syntactic watermarking (especially the possibility of modifying a document to remove the watermarks), but “the watermarked document is different from the original document!!!” is not one of them. “The original document” was only ever a cloud of probabilities in the first place, before you sampled a particular point from the cloud.
Zvi also spends a lot of time explaining this, so again, please read Zvi’s post. There are only so many times I can re-explain it over four years.
Comment #3 August 22nd, 2026 at 6:19 pm
Scott #2: Whoa! that’s pretty harsh too. I can assure you, Gruber is no ignoramus — he’s the leading, best known, and most highly respected blogger in the Apple universe. One of the few who has access to top Apple execs, and frequently gets lengthy interviews with them. Perhaps you should also take a look at his followup, which discusses the “temperature-based randomness” in this approach.
Comment #4 August 22nd, 2026 at 6:22 pm
Here’s something that confuses me: Why isn’t Anthropic using the following, far simpler scheme?
1. Every piece of text they generate is chunked using content-defined chunking (with publicly defined parameters).
2. All of the chunks are hashed, and the digests stored.
3. An API allows asking if a digest is present in the DB.
I think this could be done in a way that preserves confidentiality, even if the digests were all made public. And they could perform whitespace normalization (etc.) to make the digests more robust. But I assume I’m missing something.
Comment #5 August 22nd, 2026 at 6:28 pm
Gruber’s arguments are very weird. They mostly seem to boil down to “How could you ruin the crystalline perfection of LLM outputs?” Like… what the heck, man, it’s *randomized already*. You’re not running at temperature=0. And he seems to know that at some level, so I feel like what’s actually going on is maybe that he doesn’t want people to be able to tell when he’s using LLMs.
Now, I have seen some complaints about Anthropic’s blog post, and how it treats prose as Generic Text Product where the specifics of word choice don’t matter. And I think that would be a great criticism if we weren’t talking about an LLM. It’s already a statistical model! That’s kind of the point!
So in both cases… if you want good writing, write it yourself. 🙂
Comment #6 August 22nd, 2026 at 9:11 pm
I wonder how often they will rotate secret keys? The detection algorithm could tell you not only whether it was written by Claude or not, but approximately when it was generated.
Could they use secret keys for other purposes, too? For example, to try to detect which accounts are being used to resell their APIs to China?
Comment #7 August 22nd, 2026 at 9:23 pm
Brian Slesinsky #6: Yeah, at least the watermark could include which model it is, and maybe a few bits of additional information. A timestamp is an interesting idea as well.
Comment #8 August 22nd, 2026 at 9:28 pm
Speaking of Pangram, I recently read this piece about how that works (H/T to Ozy), and there’s one part in it I thought was really worth pointing out:
So, Pangram’s detection is premised on the fact that these have been through fine-tuning and RLHF, and aren’t just base models! Huh!
Comment #9 August 23rd, 2026 at 1:06 am
I have some concerns about agents using these techniques to communicate with each other in ways we can’t (yet) detect across security or privacy boundaries, presenting possibilities for semantic injection or other hazards.
There are also layers of metadata and traffic patterns that could be exploited for inter-agent signalling.
Comment #10 August 23rd, 2026 at 2:09 am
Tim McCormack #4:
What stops Anthropic from taking any text, computing its digest, and saving it to their DB using your scheme?
I have two main concerns about AI watermarks:
1. They don’t stop any real criminals, as it is super easy to remove them (e.g., the emojis trick).
2. They can be used for regulatory, economic, and monopolistic capture. The cartel of US AI companies has already shown their principles and “values” — in how they credit the work of others, in their piracy, in how they try to discredit open AI models, and in how they used their monetary advantage to make everyone else poorer hardware-wise. Watermarks are a new possibility for them to abuse and exploit the system.
Speaking of services that use statistics to detect AI, such as Pangram, I am afraid this could be another source of harm. This is very similar to how people use statistics to find cheating in chess. It can work quite well, except for the times when it does not. Such statistics could be a basis for abuse and harassment, which has happened in chess many times already and even led to the death of a quite prominent chess player not long ago.
I believe people should value actual results without much attention to the tools used (unless it’s the training of students or sports competitions). This is why I don’t quite support AI watermarks, especially their legislation.
Comment #11 August 23rd, 2026 at 2:34 am
Although I identify more as an accelerationist, this is an idea that seems reasonable to me.
Comment #12 August 23rd, 2026 at 3:03 am
What I don’t understand is that the detector don’t have the initial prompt, so I can’t have the same prob distribution of the text than the generator, can it ?
Let’s suppose my prompt is “Can you explain me quantum computing, but starting from the idea of a classical rotational computer (where I set up two angles in space, and get back the projection on the x axis, cos(theta)cos(phi)”
ChatGPT reply start like this :
“””
Yes. That is actually a very good route into quantum computing, because a single qubit looks remarkably like a little classical orientation in space—until we start asking what “reading it” means and what happens when we combine several of them.
Let me build it starting exactly from your hypothetical rotational computer.
1. Your classical rotational bit
Imagine a machine whose internal state is an arrow of length 1 in 3D.
(continuation)
“””
Suppose I post that as a purely novel creation of mine on my blog. I skip the prompt and the prolog :
“””
Imagine a machine whose internal state is an arrow of length 1 in 3D.
(continuation)
“””
Presumably, P(“state”|”Imagine a machine whose internal”) ≠P(“state”|Prompt + COT + “Yes. That is actually a very good route…” + “Imagine a machine whose internal”), so how does the watermark still work under that kind of change ?
Comment #13 August 23rd, 2026 at 7:29 am
AFAIK, Claude is mostly used for program code and not human-readable text generation, which is much more constrained. Does code output still have enough entropy for the watermarking to work (well enough) for it?
Comment #14 August 23rd, 2026 at 7:33 am
Joe #3: In the first article, Gruber seems to be assuming that the watermarking scheme changes the underlying probability distribution, making it less likely to choose the ‘best’ (highest probability) word. But, as I understand it:
– if you wanted the ‘best’ word at each point, you would run the inference at temperature 0. But the LLMs he likes are not doing this, because it is generally considered to produce worse outputs.
– the watermarking scheme doesn’t bias the word choice away from the ‘best’ word; it leaves the underlying probability distributions alone and just “changes the source of the randomness used to pick among words” — i.e., it leaves a detectable pattern in the sequence of choices made, but it is just as likely to bias the choices toward the ‘best’ words as away from them.
In the followup, he sort of acknowledges this, but expresses strong scepticism:
> Advocates of LLM watermarking schemes for text argue that the schemes don’t necessarily lower the quality of the generated prose, because they don’t change the temperatures — they only change the source of the randomness. Daniel Jalkut wrote a good piece today about this. I hope that’s true. I believe it’s possible that it is true. I think it’s highly unlikely that it is true. I do not see how a detectable signal can be added encoded in the choice of words without affecting the meaning of the prose. If it were true I think they’d show examples proving that it’s true.
I don’t understand why he finds it so hard to believe that a detectable signal can be encoded without affecting the meaning, except in the trivial sense: of course, if the watermarking leaves a signal at all, then the output must be different from the unwatermarked output. But we’re talking about an output that is *already* created via weighted random choices, and so the unwatermarked version could be any one of many valid versions. All the watermarking has to do is bias the output toward certain of those versions and away from others, and there’s no reason why it would have to bias the output toward worse versions and away from better ones.
Comment #15 August 23rd, 2026 at 7:43 am
I wonder, how many bits of key space are we talking about here? If there are enough free bits available, inclusion of user IDs will happen eventually, never mind the AI companies promises now.
Comment #16 August 23rd, 2026 at 7:48 am
Joe #3:
I’m with Scott (#2) on this one.
> He’s the leading, best known, and most highly respected blogger in the Apple universe. One of the few who has access to top Apple execs, and frequently gets lengthy interviews with them.
All of these facts are compatible with his being an ignoramus. And his blog post shows that he has deeply and profoundly failed to understand how the watermarking proposal works. Gruber probably also objects to the use of AES-256 encryption on the grounds that it isn’t a one-time pad, and he deserves “only the best” encryption protocol.
Where he really lost me was when he said “The idea that anything other than my needs should factor into the generation of text for me is patently offensive.” This statement demonstrates not only a truly comical level of self-centeredness and entitlement, but also a completely cluelessness about the many layers of modification that all of the frontier labs had already placed around their LLMs at every step of the pipeline, from training dataset curation to RLHF after pretraining to screening inputs and outputs at the point of inference – all of which have a complicated combination of positive and negative effects on the “quality” of the text, but which certainly incorporate considerations “other than my needs”.
Based on just this one blog post, Gruber is not worth reading, and his reputation is undeserved.
Comment #17 August 23rd, 2026 at 7:52 am
VZ #13: Yes, most code should have plenty of entropy. Think of variable names, whitespace, and in general, the multiple valid ways to solve the same little problem (c=a+b vs c=b+a), multiplied out by how many little problems there are.
Comment #18 August 23rd, 2026 at 7:55 am
hwold #12: Yes, in some sense that’s the main technical problem that my scheme solved! The key idea is that you’re favoring certain n-grams of tokens over others in a way that’s independent of the prompt or the probability distribution, and that can therefore be detected later given only the sequence of tokens (as well as the key of the pseudorandom function). The somewhat interesting part is how you do that while still appearing to sample from the same probability distribution.
Comment #19 August 23rd, 2026 at 8:15 am
Matt #14: Indeed. And your last sentence gets to yet another reason why Gruber’s analysis is wrong. He objects to the use of pseudorandom sampling of the space of next possible tokens, as opposed to the Platonic ideal of purely random sampling. But in reality, any production-grade LLM is almost certainly already using pseudorandom number generators to sample the next token, since in practice pRNGs are so much faster and more convenient to work with at scale than hardware RNGs are. So Gruber is really just objecting to the use of one pseudorandom sampling scheme over another one, for no reason that I can understand.
Comment #20 August 23rd, 2026 at 10:08 am
Joe #3: He may well “have access to top Apple execs,” but he doesn’t understand how LLMs sample from probability distributions, and what’s worse, he wrongly imagines that he does, and that he can dismiss a whole technical literature based on a minute’s worth of incorrect thinking.