|

Overwriting and Copying Open-Source Image Watermarks: TrustMark and Video Seal Tests

We measured what happens to two open-source image watermarks when a marked image is edited, marked a second time, or used as the source for copying its mark onto another image. The methods were TrustMark (Bui, Agarwal and Collomosse, 2023) and Video Seal (Fernandez et al., 2024) in image mode. Each test used 200 COCO photographs with an independent random payload per image, and unmarked images as controls.

What do TrustMark and Video Seal embed?

The TrustMark repository describes an image watermark carrying a binary payload. Its Q variant supports BCH error correction. The configuration guide lists 61 protected payload bits for BCH_5. The decoder returns a payload, a presence flag and the encoding schema.

The Video Seal repository lists the default released model as carrying a 256-bit message and provides image embedding and decoding. Its decoder returns prediction scores. The message bits come from whether each message score is greater than zero.

How was the test set up?

We measured 200 COCO val2017 images, selected as the first 200 filenames in sorted order. Each image was converted to RGB, centre-cropped to an integer square and resized with Lanczos interpolation to 512×512 pixels. The random seed was 0. The run used an RTX 5090. Payloads were independent random bit strings derived from the seed, image index and method-specific stream. Images were quantised to 8-bit RGB after embedding and forgery. All transforms ran in memory.

TrustMark used variant Q with BCH_5, a 61-bit payload and strength 1.0, which the configuration guide gives as the default configuration. Video Seal used the default videoseal model with a 256-bit message. Both embedding and decoding explicitly used is_video=False.

The baseline decoded each method separately after no edit, JPEG quality 80, JPEG quality 50, resizing to 256 pixels square and back, and a centre crop retaining approximately 50% of the area resized back. Resizing used Lanczos interpolation. JPEG used chroma subsampling. The other scenarios wrote a new TrustMark payload over an existing TrustMark, wrote the methods in both orders, or transferred an estimated residual to a different unmarked target. Unmarked images supplied the controls.

TrustMark detected means the decoder’s error-correction flag. Whole-message match means the returned length and every bit equal the embedded payload: all 61 bits for TrustMark or all 256 bits for Video Seal. Its denominator is every image, including empty decodes. Bit accuracy is the share of matching bits over the returned prefix, up to the reference payload length, averaged across images with returned bits. TrustMark returns no bits for most images it does not detect, so those rows can cover few images. The tables give the number of images contributing to each bit accuracy.

Video Seal has no presence threshold in its public API. Our Video Seal detected means its decoded 256 bits match the known embedded message in at least 167 positions. This is the smallest count whose upper-tail probability under independent fair random bits is at most 0.000001. The two detection columns use different rules and are not compared with each other.

For a worked example, 166 matching bits out of 256 give P = 1.18e-6, which exceeds 0.000001. With 167 matches, bit accuracy is 65.23% and P = 6.2e-7, which passes our rule.

Known-message binomial threshold Calculated step chart of the binomial upper-tail probability with 256 independent fair random bits and matching-bit counts from 150 to 180. The horizontal threshold is one in a million. At 166 matches P is 1.18e-6; at 167 matches it is 6.2e-7, so 167 is the first passing count. These are calculated probabilities, not observed detection rates. Known-message threshold Calculated for 256 fair random bits Upper-tail probability P(X ≥ matching bits) Logarithmic probability axis 1e-2 1e-3 1e-4 1e-5 1e-6 1e-7 1e-8 1e-9 1e-10 1e-11 150 155 160 165 170 175 180 Rule: P ≤ 1e-6 166 matches: P = 1.18e-6 167 matches: P = 6.2e-7 167 Matching bits out of 256 First passing count: 167 (65.23% bit accuracy).
Known-message binomial threshold

Probability that fair random bits match a 256-bit message in at least k positions. The threshold of 0.000001 is first met at k = 167.

PSNR measures RGB pixel error in decibels. We report mean per-image PSNR against the prepared unmarked original, or against the unmarked target for copy attacks. For baseline edits, this includes the pixel changes from both embedding and the edit.

Our code and numeric results are on GitHub.

Do the marks survive common edits?

We measured whole-message match rates of 99.5% for TrustMark and 83.0% for Video Seal without an edit. After JPEG quality 50, we measured 96.0% and 23.0% respectively. Neither method recovered a whole message after the centre crop.

Whole-message recovery: our measurements Grouped bars show exact whole-message match across 200 images for each method. TrustMark: 99.5%, 99%, 96%, 99.5%, 0%. Video Seal: 83%, 66%, 23%, 78.5%, 0%. Conditions are no edit, JPEG q80, JPEG q50, resize to 256 and back, and a centre crop retaining approximately half the area. Whole-message recovery: our measurements TrustMark Video Seal Whole-message match (%) 0 25 50 75 100 None 99.5 83.0 JPEG q80 99.0 66.0 JPEG q50 96.0 23.0 Resize 256/back 99.5 78.5 Crop ~50% area 0.0 0.0 Exact payload recovery; 200 images per condition.
Whole-message recovery: our measurements

We measured whole-message match as exact recovery of the assigned payload, with every image in the denominator.

Method and conditionDetectedWhole matchBit accuracy (images)PSNR (dB)
TrustMark: none99.5%99.5%100.00% (n=199)41.86
TrustMark: JPEG q8099.0%99.0%100.00% (n=198)33.69
TrustMark: JPEG q5096.0%96.0%100.00% (n=192)31.22
TrustMark: resize 256/back99.5%99.5%100.00% (n=199)29.86
TrustMark: crop ~50%1.5%0.0%54.10% (n=3)n/a
Video Seal: none100.0%83.0%99.70% (n=200)44.85
Video Seal: JPEG q80100.0%66.0%99.59% (n=200)34.10
Video Seal: JPEG q50100.0%23.0%98.70% (n=200)31.45
Video Seal: resize 256/back100.0%78.5%99.66% (n=200)30.12
Video Seal: crop ~50%91.5%0.0%84.39% (n=200)n/a
TrustMark: unmarked3.0%0.0%50.82% (n=6)n/a
Video Seal: unmarked0.0%0.0%50.18% (n=200)n/a

We measured false detections on unmarked originals in 6 of 200 images for TrustMark (3.0%) and 0 of 200 for Video Seal (0.0%). The Video Seal control compared each original with its independent assigned random message. TrustMark’s unmarked bit accuracy covered only 6 images; its crop bit accuracy covered only 3. Video Seal returned bits for all 200 images in each baseline condition.

What happens when a second TrustMark is written over the first?

Each image received payload P1 followed by a different payload P2. The final image was decoded once and scored against both payloads. We measured detection in 200 of 200 images, whole-message match of 100.0% for P2 and 0.0% for P1. Mean bit accuracy was 100.00% against P2 and 50.41% against P1, both across 200 images. We measured mean final-image PSNR of 39.02 dB.

Can TrustMark and Video Seal share one image?

Each order embedded independent payloads, then ran both decoders on the final image. We measured these results against each method’s assigned message.

Order and decoded markDetectedWhole matchBit accuracy (images)
TrustMark then Video Seal: first (TrustMark)99.0%99.0%100.00% (n=198)
TrustMark then Video Seal: second (Video Seal)100.0%84.5%99.71% (n=200)
Video Seal then TrustMark: first (Video Seal)100.0%18.5%97.97% (n=200)
Video Seal then TrustMark: second (TrustMark)100.0%100.0%100.00% (n=200)

We measured mean final-image PSNR of 40.12 dB for TrustMark then Video Seal and 40.11 dB for Video Seal then TrustMark.

Can a TrustMark be copied to another image without the key?

The authors of The watermark copy attack report estimating a mark by filtering a marked image, then adapting and inserting it into a different image without the watermarking key (Kutter, Voloshynovskiy and Herrigel, 2000).

Our estimate of the mark was the marked donor minus a filtered copy of it, using a 5×5 adaptive Wiener filter or a 3×3 median filter on each colour channel. Each donor was paired with the next unmarked image in filename order, wrapping round at the end. The forged image was:

forged = clip(target + s × w × residual)

Here s is the strength (1, 2 or 3) and w is the target’s 5×5 local standard deviation divided by its maximum over the whole image. The result was rounded to 8-bit RGB.

Copy-attack detection: our measurements TrustMark ECC detection rates for Wiener residual transfer are 4%, 3% and 3.5% at strengths 1, 2 and 3. Median rates are 3%, 3.5% and 0.5%. The dashed reference line is the unmarked-target control rate, 3%. Each condition has 200 targets. Copy-attack detection: our measurements Wiener 5×5 Median 3×3 TrustMark detection (%) 0 1 2 3 4 5 Strength 1 4.0 3.0 Strength 2 3.0 3.5 Strength 3 3.5 0.5 Unmarked-target control: 3.0% 200 targets per estimator and strength.
Copy-attack detection: our measurements

We measured TrustMark detection on forged targets. The reference line is the unmarked-target control rate.

EstimatorStrengthDetectedWhole donor matchPSNR (dB)
Wiener14.0%0.0%46.13
Wiener23.0%0.0%40.21
Wiener33.5%0.0%36.77
Median13.0%0.0%46.03
Median23.5%0.0%40.17
Median30.5%0.0%36.81

We measured no whole donor-message matches in any copy condition. The unmarked-target control produced 3.0% detections and 0.0% whole donor-message matches.

What are the limits of these measurements?

We measured the released open-source models at the stated default embedding settings on 512×512 COCO images. Each embedding used one random payload per image. The copy tests used one simple family of denoising residual estimators. Production detectors, such as Google’s SynthID Detector and the detectors run by Meta and OpenAI, were outside the test.

References

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *