Overwriting and Copying Open-Source Image Watermarks: TrustMark and Video Seal Tests
We measured what happens to two open-source image watermarks when a marked image is edited, marked a second time, or used as the source for copying its mark onto another image. The methods were TrustMark (Bui, Agarwal and Collomosse, 2023) and Video Seal (Fernandez et al., 2024) in image mode. Each test used 200 COCO photographs with an independent random payload per image, and unmarked images as controls.
What do TrustMark and Video Seal embed?
The TrustMark repository describes an image watermark carrying a binary payload. Its Q variant supports BCH error correction. The configuration guide lists 61 protected payload bits for BCH_5. The decoder returns a payload, a presence flag and the encoding schema.
The Video Seal repository lists the default released model as carrying a 256-bit message and provides image embedding and decoding. Its decoder returns prediction scores. The message bits come from whether each message score is greater than zero.
How was the test set up?
We measured 200 COCO val2017 images, selected as the first 200 filenames in sorted order. Each image was converted to RGB, centre-cropped to an integer square and resized with Lanczos interpolation to 512×512 pixels. The random seed was 0. The run used an RTX 5090. Payloads were independent random bit strings derived from the seed, image index and method-specific stream. Images were quantised to 8-bit RGB after embedding and forgery. All transforms ran in memory.
TrustMark used variant Q with BCH_5, a 61-bit payload and strength 1.0, which the configuration guide gives as the default configuration. Video Seal used the default videoseal model with a 256-bit message. Both embedding and decoding explicitly used is_video=False.
The baseline decoded each method separately after no edit, JPEG quality 80, JPEG quality 50, resizing to 256 pixels square and back, and a centre crop retaining approximately 50% of the area resized back. Resizing used Lanczos interpolation. JPEG used chroma subsampling. The other scenarios wrote a new TrustMark payload over an existing TrustMark, wrote the methods in both orders, or transferred an estimated residual to a different unmarked target. Unmarked images supplied the controls.
TrustMark detected means the decoder’s error-correction flag. Whole-message match means the returned length and every bit equal the embedded payload: all 61 bits for TrustMark or all 256 bits for Video Seal. Its denominator is every image, including empty decodes. Bit accuracy is the share of matching bits over the returned prefix, up to the reference payload length, averaged across images with returned bits. TrustMark returns no bits for most images it does not detect, so those rows can cover few images. The tables give the number of images contributing to each bit accuracy.
Video Seal has no presence threshold in its public API. Our Video Seal detected means its decoded 256 bits match the known embedded message in at least 167 positions. This is the smallest count whose upper-tail probability under independent fair random bits is at most 0.000001. The two detection columns use different rules and are not compared with each other.
For a worked example, 166 matching bits out of 256 give P = 1.18e-6, which exceeds 0.000001. With 167 matches, bit accuracy is 65.23% and P = 6.2e-7, which passes our rule.
Probability that fair random bits match a 256-bit message in at least k positions. The threshold of 0.000001 is first met at k = 167.
PSNR measures RGB pixel error in decibels. We report mean per-image PSNR against the prepared unmarked original, or against the unmarked target for copy attacks. For baseline edits, this includes the pixel changes from both embedding and the edit.
Our code and numeric results are on GitHub.
Do the marks survive common edits?
We measured whole-message match rates of 99.5% for TrustMark and 83.0% for Video Seal without an edit. After JPEG quality 50, we measured 96.0% and 23.0% respectively. Neither method recovered a whole message after the centre crop.
We measured whole-message match as exact recovery of the assigned payload, with every image in the denominator.
| Method and condition | Detected | Whole match | Bit accuracy (images) | PSNR (dB) |
|---|---|---|---|---|
| TrustMark: none | 99.5% | 99.5% | 100.00% (n=199) | 41.86 |
| TrustMark: JPEG q80 | 99.0% | 99.0% | 100.00% (n=198) | 33.69 |
| TrustMark: JPEG q50 | 96.0% | 96.0% | 100.00% (n=192) | 31.22 |
| TrustMark: resize 256/back | 99.5% | 99.5% | 100.00% (n=199) | 29.86 |
| TrustMark: crop ~50% | 1.5% | 0.0% | 54.10% (n=3) | n/a |
| Video Seal: none | 100.0% | 83.0% | 99.70% (n=200) | 44.85 |
| Video Seal: JPEG q80 | 100.0% | 66.0% | 99.59% (n=200) | 34.10 |
| Video Seal: JPEG q50 | 100.0% | 23.0% | 98.70% (n=200) | 31.45 |
| Video Seal: resize 256/back | 100.0% | 78.5% | 99.66% (n=200) | 30.12 |
| Video Seal: crop ~50% | 91.5% | 0.0% | 84.39% (n=200) | n/a |
| TrustMark: unmarked | 3.0% | 0.0% | 50.82% (n=6) | n/a |
| Video Seal: unmarked | 0.0% | 0.0% | 50.18% (n=200) | n/a |
We measured false detections on unmarked originals in 6 of 200 images for TrustMark (3.0%) and 0 of 200 for Video Seal (0.0%). The Video Seal control compared each original with its independent assigned random message. TrustMark’s unmarked bit accuracy covered only 6 images; its crop bit accuracy covered only 3. Video Seal returned bits for all 200 images in each baseline condition.
What happens when a second TrustMark is written over the first?
Each image received payload P1 followed by a different payload P2. The final image was decoded once and scored against both payloads. We measured detection in 200 of 200 images, whole-message match of 100.0% for P2 and 0.0% for P1. Mean bit accuracy was 100.00% against P2 and 50.41% against P1, both across 200 images. We measured mean final-image PSNR of 39.02 dB.
Can TrustMark and Video Seal share one image?
Each order embedded independent payloads, then ran both decoders on the final image. We measured these results against each method’s assigned message.
| Order and decoded mark | Detected | Whole match | Bit accuracy (images) |
|---|---|---|---|
| TrustMark then Video Seal: first (TrustMark) | 99.0% | 99.0% | 100.00% (n=198) |
| TrustMark then Video Seal: second (Video Seal) | 100.0% | 84.5% | 99.71% (n=200) |
| Video Seal then TrustMark: first (Video Seal) | 100.0% | 18.5% | 97.97% (n=200) |
| Video Seal then TrustMark: second (TrustMark) | 100.0% | 100.0% | 100.00% (n=200) |
We measured mean final-image PSNR of 40.12 dB for TrustMark then Video Seal and 40.11 dB for Video Seal then TrustMark.
Can a TrustMark be copied to another image without the key?
The authors of The watermark copy attack report estimating a mark by filtering a marked image, then adapting and inserting it into a different image without the watermarking key (Kutter, Voloshynovskiy and Herrigel, 2000).
Our estimate of the mark was the marked donor minus a filtered copy of it, using a 5×5 adaptive Wiener filter or a 3×3 median filter on each colour channel. Each donor was paired with the next unmarked image in filename order, wrapping round at the end. The forged image was:
forged = clip(target + s × w × residual)
Here s is the strength (1, 2 or 3) and w is the target’s 5×5 local standard deviation divided by its maximum over the whole image. The result was rounded to 8-bit RGB.
We measured TrustMark detection on forged targets. The reference line is the unmarked-target control rate.
| Estimator | Strength | Detected | Whole donor match | PSNR (dB) |
|---|---|---|---|---|
| Wiener | 1 | 4.0% | 0.0% | 46.13 |
| Wiener | 2 | 3.0% | 0.0% | 40.21 |
| Wiener | 3 | 3.5% | 0.0% | 36.77 |
| Median | 1 | 3.0% | 0.0% | 46.03 |
| Median | 2 | 3.5% | 0.0% | 40.17 |
| Median | 3 | 0.5% | 0.0% | 36.81 |
We measured no whole donor-message matches in any copy condition. The unmarked-target control produced 3.0% detections and 0.0% whole donor-message matches.
What are the limits of these measurements?
We measured the released open-source models at the stated default embedding settings on 512×512 COCO images. Each embedding used one random payload per image. The copy tests used one simple family of denoising residual estimators. Production detectors, such as Google’s SynthID Detector and the detectors run by Meta and OpenAI, were outside the test.
References
- Bui, T., Agarwal, S., and Collomosse, J., “TrustMark: Universal Watermarking for Arbitrary Resolution Images,” 30 November 2023.
- Adobe, “TrustMark,” undated.
- Fernandez, P., Elsahar, H., Yalniz, I. Z., and Mourachko, A., “Video Seal: Open and Efficient Video Watermarking,” 12 December 2024.
- Meta, “Video Seal,” undated.
- Kutter, M., Voloshynovskiy, S., and Herrigel, A., “The watermark copy attack,” Proc. SPIE 3971, January 2000.
- Lin, T.-Y., et al., “Microsoft COCO: Common Objects in Context,” 2014.
- Our code and numeric results: GitHub