Benchmarks

An open leaderboard comparing watermarking methods under one fixed dataset, attack suite, and set of metrics — so results are directly comparable and reproducible.

Status

🚧 Image Watermark Leaderboard v1.0 is in preparation.

Tracks

• Track A — Image (post-hoc): v1.0
• Track B — Image (in-generation): planned
• Track C — Text (LLM): planned

Protocol (draft)

• Dataset: fixed subsets of COCO val2017 (1,000 images) and DIV2K validation (100), center-cropped to 512×512; image IDs and hashes published
• Payload: 32 bits, fixed seeds
• Quality: PSNR, SSIM, LPIPS — methods compared at a matched quality operating point
• Robustness: bit accuracy and TPR@1% FPR under compression, geometric, signal-processing, combined, and regeneration attacks
• Practicality: embed/extract time per image

Reproducibility

Evaluation code, configs, and raw results will be published on GitHub: [repo link]. New methods can be submitted via pull request.