2026-08-18 06:57 UTC
John Gruber's article responding to Anthropic's watermarking is silly, and he deserves to be dunked on. I just hope people stay vigilant against assuming that the system is good because LLM users are against it. I don't use LLMs, and I'd like to have a reliable system for detecting their output, but unfortunately I don't think that's a realistic possibility.
Remember that the claims about the efficacy of this approach are coming from a company run by people who believe their statistical model has emotions. Their core product is held together with successive layers of duct tape and band-aids, and we know this because its source code was leaked due to poor security practices. Oh, and let's not forget that these people lie constantly in addition to being incompetent.
The watermarking approach they intend to use is a lot like mandatory online age verification in that:
it is unreliableit has negative externalities
It's also a bit like the key escrow systems proposed in the 1990s in that they will need to safely distribute a single shared key to every one of their datacenters. If (when) a key leaks they will need to switch to using a new key for generation, and check against all previous keys at the detection phase.
The paper this is based on has "scalable watermarking" in the title, but that's only in comparison to the approach of checking text against a giant database of all previously generated text. More realistically, it scales very poorly in a bunch of ways, and will only really serve to create additional demand for datacenters.
Our environment and electricity bills are already bearing the costs of generation, but now we'll have people introducing an extra watermark-stripping step, and then more detection steps which (I can't stress enough) are not going to work.
Replies (1)
-
@davidgerard@circumstances.run 2026-08-18 09:42
@ansuz@gts.cryptography.dog the Google LLM watermarking system, the one Anthropic uses, is the sort of system that's an interesting paper, requires a ton of text, has a ghastly rate of false positive and negatives and obeys the EU rules to the "you treid" standard. But it is apparently good enough the chatbots can avoid drinking too much of their own piss, which is the obvious reason Google bothered.