Watermarking LLM Text: A Signal Hidden in the Sampling

A language model doesn’t choose its next token; it gives a distribution, and a sampler draws from it (Decoding). At most steps several tokens are reasonable: “the”, “a” or “this”; “big” or “large”. That slack is where a watermark can live. If the model’s provider nudges the choice, in a way only it knows, toward a hidden subset of tokens, the text it produces carries a statistical signature. Anyone with the key can test for it, without the model, the prompt, or any other text; to anyone else the text reads normally.

This post derives the best-known scheme and its test, runs it on a real model, and measures what it costs and what removes it.

Table of contents

The green list

Kirchenbauer et al. (2023) proposed the scheme most others build on. Before each token is sampled:

1. Hash the previous token together with a secret key, and use the result to seed a random number generator.
2. With it, split the vocabulary at random: a fraction $\gamma$ of tokens is the green list, the rest red.
3. Add a constant $\delta$ to every green token’s logit, then sample as usual.

Adding $\delta$ to a logit multiplies that token’s unnormalized probability by $e^\delta$, so with $\delta = 2$ every green token becomes 7.4 times as likely relative to the red ones. When the model is sure of the next token (the second half of “United States”, the closing bracket of an expression), one token holds almost all the probability, and a bias of 2 hardly changes the choice: quality is protected exactly where it matters. When the model is unsure, many tokens are close, and the bias tips the choice toward green ones.

The split changes at every position, because it is seeded by the previous token. Without the key, a single text shows no visible pattern: no favorite words, nothing a reader could spot. Over many texts it does leave one. Since the list depends only on the previous token, the same pair of tokens is always green or always red, so the watermarked model’s bigram frequencies are skewed. An attacker who collects enough of its output can estimate the green lists and then strip the watermark, or forge it onto text the model never wrote (watermark stealing; Jovanović et al., 2024). Seeding from a longer context, the previous $h$ tokens instead of one, makes the lists harder to learn, at the cost of robustness: an edit then breaks the lists of the next $h$ tokens, not just one.

Detection: a z-test

Detection needs the text, the tokenizer and the key. Recompute the green list at each position from the token before it, and count how many tokens are green. If the text was written without the watermark, each token is green with probability $\gamma$, independently of the others, because whoever wrote it didn’t know the lists. So in $T$ tokens the green count $|s|_G$ has mean $\gamma T$ and variance $T\gamma(1-\gamma)$, and

is approximately standard normal. Watermarked text has more green tokens than chance; a large $z$ rejects the hypothesis that it was written without the key. Kirchenbauer et al. use $z > 4$, whose false-positive rate, from the normal tail, is $3 \times 10^{-5}$. Two caveats, both measured below. The green count is really binomial, whose tail at these lengths is one and a half to four times fatter than the normal one. And “independently” fails when a (previous token, token) pair repeats in a text, since it repeats the same outcome; the fix is to score each distinct pair only once.

Worked, for $\gamma = 0.25$, $\delta = 2$, $T = 200$:

import math
# Detection is a one-proportion z-test. Without a watermark each scored
# token is green with probability gamma, independently (the list depends on
# a secret key), so in T tokens the green count has mean gamma T and
# variance T gamma (1 - gamma):
#   z = (greens - gamma T) / sqrt(T gamma (1 - gamma)).
gamma, delta, T = 0.25, 2.0, 200
# If the model were indifferent between all tokens, adding delta to the
# green logits would make a token green with probability
#   gamma e^delta / (gamma e^delta + 1 - gamma),
# the most the watermark can do. Low-entropy text gives less.
g_max = gamma * math.exp(delta) / (gamma * math.exp(delta) + 1 - gamma)
z = lambda g, T: (g - gamma) * T / math.sqrt(T * gamma * (1 - gamma))
print(f"gamma = {gamma}, delta = {delta}: a flat distribution's green share is {g_max:.3f}")
print(f"z for T = {T} tokens at that share: {z(g_max, T):.1f}")
print(f"\n{'green share':>11s} {'z at T=50':>10s} {'z at T=200':>11s} {'tokens for z = 4':>17s}")
for g in (0.30, 0.35, 0.40, 0.50, g_max):
    need = 16 * gamma * (1 - gamma) / (g - gamma) ** 2
    print(f"{g:11.3f} {z(g, 50):10.1f} {z(g, 200):11.1f} {math.ceil(need):17d}")
# A false positive needs z > 4 on unwatermarked text. The normal tail is the
# large-T limit; the green count is really binomial, with a fatter tail.
print(f"\nfalse-positive rate at z > 4: normal tail {0.5 * math.erfc(4 / math.sqrt(2)):.1e}", end="")
for T in (50, 200, 1000):
    k = math.floor(gamma * T + 4 * math.sqrt(T * gamma * (1 - gamma))) + 1      # fewest greens with z > 4
    exact = sum(math.comb(T, i) * gamma ** i * (1 - gamma) ** (T - i) for i in range(k, T + 1))
    print(f"; exact at T = {T}: {exact:.1e}", end="")
print()

Full script: z_test.py

gamma = 0.25, delta = 2.0: a flat distribution's green share is 0.711
z for T = 200 tokens at that share: 15.1

green share  z at T=50  z at T=200  tokens for z = 4
      0.300        0.8         1.6              1201
      0.350        1.6         3.3               301
      0.400        2.4         4.9               134
      0.500        4.1         8.2                48
      0.711        7.5        15.1                15

false-positive rate at z > 4: normal tail 3.2e-05; exact at T = 50: 1.2e-04; exact at T = 200: 6.2e-05; exact at T = 1000: 4.8e-05

If the model were indifferent among all tokens, the bias would make 71% of tokens green and $z$ would reach 15 in 200 tokens. Real text is far from indifferent, and the green share depends on how much uncertainty the text left. At a 40% green share, 200 tokens give $z = 4.9$ on average; at 30%, the average $z$ reaches 4 only after about 1,200 tokens. Those are expected values: at the length where the average $z$ is 4, about half of texts are detected, so reliable detection needs more. The $\sqrt{T}$ in the numerator is the general rule: the signal per token is fixed, and evidence grows with the square root of length.

Try it, with a synthetic model whose confidence can be adjusted:

On a real model

The scheme on Qwen2.5-1.5B-Instruct: 10 prompts, 200 tokens each, temperature 0.7, $\gamma = 0.25$, four bias strengths. Quality is measured as the average negative log-likelihood of the generated tokens under the unwatermarked distribution, which goes up as the bias pushes the model toward tokens it rated less likely. The green list, the detector and the watermarked sampler, from wm.py:

def green(prev):
    """The green list for a given previous token: a keyed permutation of the
    vocabulary, of which the first gamma V tokens are green."""
    if prev not in _green:
        g = torch.Generator().manual_seed(KEY * (prev + 1) % (2 ** 63 - 1))
        mask = torch.zeros(V, dtype=torch.bool)
        mask[torch.randperm(V, generator=g)[: int(GAMMA * V)]] = True
        _green[prev] = mask
    return _green[prev]

def z_score(ids):
    """ids: the scored tokens, preceded by the one token before them."""
    hits = sum(bool(green(p)[t]) for p, t in zip(ids[:-1], ids[1:]))
    n = len(ids) - 1
    return (hits - GAMMA * n) / math.sqrt(n * GAMMA * (1 - GAMMA)), hits / n

@torch.no_grad()
def generate(prompt, delta, temp=0.7, seed=0):
    """Sample N tokens with the watermark; also return each token's
    log-probability and the entropy under the unwatermarked distribution."""
    torch.manual_seed(seed)
    ids = tok.apply_chat_template([{"role": "user", "content": prompt}], add_generation_prompt=True,
                                  return_tensors="pt", return_dict=True)["input_ids"].to(DEVICE)
    out = model(ids, use_cache=True)
    past, logits, prev = out.past_key_values, out.logits[0, -1].float().cpu(), ids[0, -1].item()
    toks, nll, ent = [prev], [], []
    for _ in range(N):
        logp = (logits / temp).log_softmax(-1)
        biased = logits / temp + delta * green(prev).float()
        t = torch.multinomial(biased.softmax(-1), 1).item()
        nll.append(-logp[t].item()); ent.append(-(logp.exp() * logp).sum().item())
        toks.append(t); prev = t
        out = model(torch.tensor([[t]], device=DEVICE), past_key_values=past, use_cache=True)
        past, logits = out.past_key_values, out.logits[0, -1].float().cpu()
    return toks, float(np.mean(nll)), float(np.mean(ent))

Full script: wm.py

The rest of wm.py generates the 10 texts at each $\delta$, scores them, and attacks the $\delta = 2$ texts by replacing tokens at random and by paraphrasing them with the same model.

gamma = 0.25, 200 tokens per text, temperature 0.7, 10 prompts
delta  green share   z (mean, min)  detected z>4  NLL (nats/token)  entropy
    0        0.247     -0.1  -1.5         0/10             0.713     0.69
    1        0.312      2.0  -0.3         1/10             0.768     0.70
    2        0.377      4.1   2.0         5/10             0.914     0.79
    4        0.550      9.8   4.9        10/10             1.410     0.82

delta = 2, texts from lowest to highest entropy:
  entropy  0.44  0.49  0.76  0.83  0.84  0.87  0.89  0.90  0.95  0.97
  z         4.1   3.4   3.8   2.0   5.4   3.9   5.7   2.9   5.9   4.4
  correlation 0.27

attacks on the delta = 2 texts: mean z (texts still detected at z > 4)
  replace 10% of tokens at random: z   3.4 (3/10)
  replace 20% of tokens at random: z   2.4 (1/10)
  replace 30% of tokens at random: z   2.0 (0/10)
  replace 50% of tokens at random: z   1.1 (0/10)
  paraphrase by the same model, unwatermarked: z   2.5 (2/10)

(The bias is added to the logits after dividing by the temperature, so $\delta = 2$ here is a bias of 1.4 on the raw logits; the 7.4× ratio above holds for the tempered distribution.)

The trade between detection and quality is steep. Without the watermark ($\delta = 0$) the green share is 24.7% and no text is flagged, as it should be. At $\delta = 2$ the green share rises to 37.7% and 5 of the 10 texts reach $z > 4$ in 200 tokens (with ten texts, the true rate could be anywhere from about a fifth to four-fifths); at $\delta = 4$, all ten are detected, but the average negative log-likelihood of the chosen tokens doubles, from 0.71 to 1.41 nats: the model is writing things it considered much less likely. Whether readers would notice is a separate question this measurement can’t answer; Kirchenbauer et al. scored quality with a larger model’s perplexity, and SynthID-Text with user ratings.

The model gave the watermark little to work with. At temperature 0.7 its distributions averaged about 0.7 nats of entropy per token, so a bias of 2 moved the green share to 38%, not the 71% that a flat distribution would allow. (The entropy column rises with $\delta$ because the bias steers the text into less predictable places; it is an effect of the watermark, not evidence about it.) An average hides the mix, and the mechanism shows when each step is grouped by its own entropy. Regenerating the same $\delta = 2$ texts with a log of every step:

steps, texts = [], []
for i, p in enumerate(prompts):
    toks, s = generate_logged(p, 2.0, seed=i)
    texts.append(toks); steps += s
ent, grn = np.array([e for e, _ in steps]), np.array([g for _, g in steps])
print(f"delta = 2: {len(steps)} steps, green rate by the step's entropy (nats)")
for lo, hi in ((0, 0.1), (0.1, 0.5), (0.5, 1.0), (1.0, 2.0), (2.0, 99)):
    m = (ent >= lo) & (ent < hi)
    print(f"  entropy {lo:3.1f} to {hi if hi < 99 else 'max':>3}: {m.sum():5d} steps ({m.mean():4.0%}), green {grn[m].mean():.0%}")

Full script: entropy_paraphrase.py

generate_logged is generate from wm.py with a record, at every step, of the entropy and of whether the chosen token was green. The script also measures how much each paraphrase copies its original.

delta = 2: 2000 steps, green rate by the step's entropy (nats)
  entropy 0.0 to 0.1:   482 steps ( 24%), green 28%
  entropy 0.1 to 0.5:   368 steps ( 18%), green 27%
  entropy 0.5 to 1.0:   462 steps ( 23%), green 37%
  entropy 1.0 to 2.0:   539 steps ( 27%), green 50%
  entropy 2.0 to max:   149 steps (  7%), green 55%

paraphrases: share of their token pairs that also occur in the original
  text 0: 78% shared, z  1.5, starts 'Parks have been a cornerstone of'
  text 1: 96% shared, z  2.4, starts 'A morning at a fishing village is'
  text 2: 76% shared, z  2.4, starts 'A bicycle remains upright based on the'
  text 3: 100% shared, z  2.0, starts 'The sun was shining brightly, casting'
  text 4: 62% shared, z  1.8, starts 'There are various qualities that make a'
  text 5: 74% shared, z  3.1, starts 'The history of bread is a captivating'
  text 6: 44% shared, z  2.1, starts "Dear [Friend's Name],\n\nI"
  text 7: 30% shared, z  0.7, starts 'the sky appears blue during the day'
  text 8: 78% shared, z  4.2, starts 'Introducing our new wooden desk lamp'
  text 9: 59% shared, z  4.6, starts 'From a high vantage point,'

A quarter of the steps have almost no uncertainty, and on them the green rate stays near the 25% of chance: the bias can’t move a token that holds nearly all the probability. Above 1 nat, half the tokens are green. That is why Kirchenbauer et al. state their detection bound in terms of a per-step measure of how spread out the distribution is (their spike entropy), and why code and other low-entropy text carry a weak watermark. Per text, the link is weaker (correlation 0.27 across ten texts), and 200 tokens is short: if the green share held, doubling the length to 400 tokens would lift the average $z$ at $\delta = 2$ from 4.1 to about 5.8.

Human text stays at chance. The detector was run with the same key over every 200-token stretch of this blog’s own posts. human.py takes the green list and the z-test from wm.py, and loads only the tokenizer:

def z_unique(ids):
    """The same test, scoring each (previous token, token) pair only once:
    a repeated pair repeats the same green outcome, so it isn't new evidence."""
    pairs = list(dict.fromkeys(zip(ids[:-1], ids[1:])))
    hits = sum(bool(green(p)[t]) for p, t in pairs)
    return (hits - GAMMA * len(pairs)) / math.sqrt(len(pairs) * GAMMA * (1 - GAMMA))

zs, zu = [], []
for path in posts:
    body = re.sub(r"```.*?```|\$\$.*?\$\$|^import .*?$|<[^>]+>", " ", open(path).read().split("---", 2)[-1], flags=re.S | re.M)
    ids = tok(" ".join(body.split())).input_ids
    for start in range(0, len(ids) - N - 1, N):
        zs.append(z_score(ids[start:start + N + 1])[0]); zu.append(z_unique(ids[start:start + N + 1]))

Full script: human.py

1135 windows of 200 tokens from 52 posts; a standard normal has mean 0, std 1
every pair         z mean -0.20 (standard error 0.033), std 1.11, max 3.92; above 4: 0; above 2: 2.6% (normal: 2.3%)
unique pairs only  z mean -0.16 (standard error 0.029), std 0.99, max 3.03; above 4: 0; above 2: 1.5% (normal: 2.3%)

None of 1,135 windows passes $z > 4$. Scoring every pair, though, the $z$ values spread more than a standard normal (std 1.11), because repeated pairs count the same evidence twice; scoring each distinct pair once brings the spread to 0.99. The mean is still below zero, by more than its standard error allows. That turns out to be a property of the key:

import numpy as np
# Is the human-text mean below 0 a property of the scheme or of one key?
# The same windows, unique pairs only, scored with five other keys.
src = open("human.py").read()
exec(src.split("print(f\"{len(zs)} windows")[0])                     # windows, green(), z_unique() for wm.py's key
windows = []
for path in posts:
    body = re.sub(r"```.*?```|\$\$.*?\$\$|^import .*?$|<[^>]+>", " ", open(path).read().split("---", 2)[-1], flags=re.S | re.M)
    ids = tok(" ".join(body.split())).input_ids
    windows += [ids[s:s + N + 1] for s in range(0, len(ids) - N - 1, N)]
for key in (15485863, 1, 2, 3, 4, 5):
    KEY = key; _green.clear()
    z = np.array([z_unique(w) for w in windows])
    print(f"key {key:>9d}: mean z {z.mean():5.2f}, std {z.std():.2f}, above 4: {(z > 4).sum()}", flush=True)

Full script: other_keys.py

key  15485863: mean z -0.16, std 0.99, above 4: 0
key         1: mean z -0.03, std 0.97, above 4: 0
key         2: mean z -0.05, std 0.98, above 4: 0
key         3: mean z -0.24, std 0.97, above 4: 0
key         4: mean z  0.29, std 0.94, above 4: 0
key         5: mean z  0.02, std 1.02, above 4: 0

With any one key, the most common token pairs of English (“of the”, ”, and”) are either green or red for every text, so all the texts from one corpus share a small offset, here between −0.24 and +0.29 depending on the key. Over random keys it averages out, which is what the false-positive guarantee is about. These 1,135 windows can show the test behaves sensibly on real prose; they can’t confirm a rate of $3 \times 10^{-5}$, which would need hundreds of thousands.

Edits erode it predictably. Replacing a token breaks two green tests: its own, and the next token’s, whose green list was seeded by it. A test survives only if neither of its two tokens was replaced, so replacing a fraction $\varepsilon$ of tokens leaves a share $(1 - \varepsilon)^2$ of tests intact, and $z$ should shrink by that factor: from 4.1 to 3.3, 2.6, 2.0 and 1.0 at 10, 20, 30 and 50%, against 3.4, 2.4, 2.0 and 1.1 measured. Paraphrasing with the same model, without the watermark, left an average $z$ of 2.5, with 2 of 10 texts still detected. The detail run above shows why so much survived: the paraphrases share 30 to 100% of their token pairs with the originals, and one simply copied its input. A 1.5B model is a weak paraphraser. Either attack cut detections from 5 of 10 to 1 or 2, while keeping about 60% of the average $z$. Longer texts resist better: in Kirchenbauer et al.’s follow-up study, even after strong paraphrasing by people, the watermark was detectable after about 800 tokens on average at a false-positive rate of $10^{-5}$, because paraphrases keep fragments of the original.

Distortion-free watermarks

The green list changes the distribution: at $\delta = 2$ the model writes measurably less likely text. A second family of schemes doesn’t change it, in the sense that, averaged over keys, every token is an exact sample from the model’s distribution, much as speculative decoding (Speculative Decoding, Derived) preserves it. They make the randomness of the sampler, not the distribution, depend on the key.

Aaronson’s scheme (2022, described in talks and later analyzed in papers) uses the Gumbel-max trick. For each candidate token $v$, the key and the preceding tokens seed a pseudo-random number $r_v$ uniform on $[0,1]$, and the sampler picks the token maximizing $r_v^{1/p_v}$. Over the randomness of $r$, that choice is distributed exactly as $p$: it’s a standard way of sampling from a distribution. But given the key, it is a deterministic function of the context, and the chosen tokens tend to have large $r_v$. The detector, which can recompute the $r$‘s, sums $-\log(1 - r)$ over the text; without the watermark each term averages 1, with it they average more.

SynthID-Text (Dathathri et al., 2024, Nature), deployed in Google’s Gemini, uses tournament sampling: draw several candidate tokens from the model’s distribution, then run them through a knockout tournament of several rounds, each round scoring candidates with a keyed pseudo-random function and keeping the winners. The winner is a sample from (nearly) the model’s own distribution, but biased toward tokens that score high under the key, which the detector tests for. Google reported, from a live experiment on about 20 million Gemini responses, no detectable difference in user ratings between watermarked and unwatermarked responses.

The catch is determinism. For one fixed key, the same context always yields the same token, so two requests with the same prompt would give the same answer. Schemes work around it: Kuditipudi et al. (2023) read the key’s random numbers from a long sequence starting at a random offset for each response, which restores variety and makes the watermark survive edits (the detector searches over offsets); SynthID-Text skips the watermark on contexts it has already seen in a response. Every scheme trades detectability, quality, diversity and robustness against each other. At the far end, cryptographic constructions (Christ et al., 2023) make watermarked text provably indistinguishable from unwatermarked text without the key.

What a watermark is not

A watermark lets the provider (or anyone it gives the key to) show, with a known false-positive rate, that its model very likely wrote a text, as long as nobody has learned to forge its signature. It says nothing about text from other models, and it is only as strong as the provider’s choice to apply it. That makes it a different thing from the AI-text detectors offered as products, which look at a text after the fact and guess from its style or its perplexity under some model whether a machine wrote it. Those have no secret to test for, so they have no false-positive guarantee like the $3 \times 10^{-5}$ above: their errors depend on the writer, and they have been shown to flag non-native English writers’ text as AI-generated far more often than native speakers’ (Liang et al., 2023). A $z$-score against a known key is evidence; a detector’s score is a guess.

Watermarks also don’t survive a determined effort to remove them on short texts, as the attacks above show. They’re best thought of as a cheap way to label most text from a provider most of the time, for tracing and for filtering model output out of future training data, not as proof against an adversary.

Where this leads

Diffusion, for Images and for Text generates text many tokens at once and in any order, so the previous token that seeds this post’s green list may not be decided yet when a token is chosen. Seeing with Transformers turns from text to images, where a model reads pixels as tokens.

Takeaways

• A green-list watermark adds $\delta$ to the logits of a keyed random fraction $\gamma$ of the vocabulary, reseeded from the previous token at every step.
• Detection counts green tokens: $z = (|s|_G - \gamma T)/\sqrt{T\gamma(1-\gamma)}$, with no access to the model. At $z > 4$, false positives are $3 \times 10^{-5}$.
• Evidence grows as $\sqrt{T}$, and the signal per token depends on the model’s uncertainty: on near-certain steps the green rate stayed at chance, above 1 nat of entropy it doubled.
• Score each distinct token pair once, and expect a fixed key to shift a corpus’s $z$ slightly; the false-positive guarantee is over keys.
• On a 1.5B model at temperature 0.7, $\delta = 2$ raised the green share to 38% and detected half of 200-token texts; $\delta = 4$ detected all of them but doubled the per-token negative log-likelihood. Replacing 20% of tokens, or a weak paraphrase, cut detections to 1 or 2 of 10 while keeping about 60% of the average $z$.
• Distortion-free schemes (Gumbel sampling, SynthID-Text’s tournament) keep the model’s distribution and hide the key in the sampler’s randomness instead.
• A watermark proves provenance for one provider’s model with a known error rate; post-hoc AI-text detectors guess, and their errors fall unevenly on different writers.

References

Start with Kirchenbauer et al. for the green-list scheme and its test, then their follow-up on reliability under paraphrase, and Dathathri et al. for a distortion-free scheme deployed at scale.

• Kirchenbauer et al., A Watermark for Large Language Models (ICML 2023): the green-list scheme, its z-test and the spike-entropy bound.
• Aaronson, My AI Safety Lecture for UT Effective Altruism (2022): the Gumbel-max watermark, as first described.
• Dathathri et al., Scalable watermarking for identifying large language model outputs (Nature, 2024): SynthID-Text’s tournament sampling and the live Gemini experiment.
• Kuditipudi et al., Robust Distortion-free Watermarks for Language Models (2023): random offsets into a key sequence, for variety and robustness to edits.
• Kirchenbauer et al., On the Reliability of Watermarks for Large Language Models (2023): detection after human and machine paraphrase, by text length.
• Jovanović, Staab and Vechev, Watermark Stealing in Large Language Models (ICML 2024): learning a key’s green lists from outputs, to forge or remove the watermark.
• Christ, Gunn and Zamir, Undetectable Watermarks for Language Models (2023): the cryptographic construction, indistinguishable without the key.
• Fernandez et al., Three Bricks to Consolidate Watermarks for Large Language Models (2023): sound statistical tests for watermark detection, including scoring each repeated token tuple once.
• Liang et al., GPT detectors are biased against non-native English writers (2023): post-hoc detectors’ false positives on non-native writers.