More Haste, Less Speed: Weaker Single-Layer Watermark Improves Distortion-Free Watermark Ensembles

Watermarking has emerged as a crucial technique for detecting and attributing content generated by large language models. While recent advancements have utilized watermark ensembles to enhance robustness, prevailing methods typically prioritize maximizing the strength of the watermark at every individual layer. In this work, we identify a critical limitation in this "stronger-is-better" approach: strong watermarks significantly reduce the entropy of the token distribution, which paradoxically weakens the effectiveness of watermarking in subsequent layers. We theoretically and empirically show that detectability is bounded by entropy and that watermark ensembles induce a monotonic decrease in both entropy and the expected green-list ratio across layers. To address this inherent trade-off, we propose a general framework that utilizes weaker single-layer watermarks to preserve the entropy required for effective multi-layer ensembling. Empirical evaluations demonstrate that this counter-intuitive strategy mitigates signal decay and consistently outperforms strong baselines in both detectability and robustness.

Key Contributions

• Theoretical proof that detectability in distortion-free watermarks is bounded by token distribution entropy, and that watermark ensembles cause monotonic entropy decay across layers
• Identification of the 'stronger-is-better' fallacy in watermark ensembles: stronger single-layer watermarks reduce entropy and paradoxically degrade multi-layer ensemble performance
• A general framework using a mixing coefficient λ to weaken single-layer watermarks, preserving entropy and improving overall ensemble detectability and robustness

🛡️ Threat Analysis

Details

Similar Papers