|

TrustMark: Universal Watermarking for Arbitrary Resolution Images (Bui et al., 2023)

TrustMark embeds an identifier through small changes to an image and trains a decoder to recover it after image processing. Tu Bui, Shruti Agarwal, and John Collomosse frame this as a way to recover provenance information: records of how an image was created and changed. Their design addresses image quality, varying image resolutions, and removing a mark before replacing it. Bui et al., TrustMark: Universal Watermarking for Arbitrary Resolution Images (2023).

Why put an identifier inside the pixels?

An origin record stored alongside an image can disappear during sharing:

“However, such metadata is often stripped as assets are re-shared and distributed online.”

— Bui et al. (2023), Section 1

An embedded identifier offers a lookup key for that record. Supplement E demonstrates querying a provenance database using a recovered watermark. The hidden message is the payload; each bit holds a zero or a one. An embedder changes pixels to carry those bits. An extractor, also called a decoder, predicts them from the received image without needing the original.

What networks does TrustMark train?

The paper trains an embedder E to encode the message and an extractor X to recover it, with a differentiable noise module N between them. A discriminator supplies the adversarial image-quality loss. TrustMark-RM trains a separate KBNet remover while the embedder and extractor remain frozen.

Figure 1a and 1b: embedder, noise module, extractor, discriminator and separate KBNet removal training

Bui et al. (2023), Figure 1.

Why work at 256×256 and resize only the residual?

The networks accept a fixed input size. Enlarging their entire output would reconstruct the image’s detail from that small output. Algorithm 1 instead transfers a residual, the signed difference between the network output and its input:

  1. Save the original height and width; normalize pixels from 0–255 to [-1, 1].
  2. Resize the input to 256×256.
  3. Run the embedder with the message, or the remover, and subtract the small input from its output.
  4. Resize this residual to the original height and width.
  5. Add it to the original normalized image; clip values to [-1, 1].
  6. Rescale pixels to 0–255.

“This method requires the watermarking/removal models to be robust against resizing”

— Bui et al. (2023), Section 3.3

Figure 10 contrasts residual scaling with enlarging the entire encoded output. PSNR measures average pixel error in decibels (dB); higher means less error. Section 4.5 reports PSNR varying by only ±0.02 dB and bit accuracy by ±10⁻⁴ on average when testing DIV2K at 20%–100% of its original 2K resolution.

Figure 10: PSNR and bit accuracy across resolution ratios, with the residual-scaling PSNR curve highlighted

Bui et al. (2023), Figure 10. The highlight follows the residual-scaling quality curve.

In our Boat run at 2048×2048, the encoder’s residual at 256×256 ranged from −15.2 to +17.5 pixel levels on the 0–255 scale. Bilinear resizing to 2048×2048 spreads each working-size pixel over an 8×8 block of full-size pixels, blended with its neighbors. At pixel (904, 993), counting from zero, the input value was 155 and the resized residual was +1.12. Adding the two and dropping the fraction gave a marked value of 156.

Resolution scaling: our Boat exampleIn our Boat run at 2048×2048, the encoder's residual at 256×256 ranged from −15.2 to +17.5 pixel levels on the 0–255 scale. Bilinear resizing to 2048×2048 spreads each working-size pixel over an 8×8 block of full-size pixels, blended with its neighbors. At pixel (904, 993), counting from zero, the input value was 155 and the resized residual was +1.12. Adding the two and dropping the fraction gave a marked value of 156. Both residual panels use the same grayscale scale. The orange square selects the pixel in the enlarged neighborhood.Resolution scaling on Boat 256×256 → 2048×2048 · red channel Working residual · mean removed −17.9 0 +17.9 Residual range · pixel levels Min -15.2 Max +17.5 Resized residual · bilinear 8×8 detail One working pixel spans 8×8 output pixels. Values are pixel levels on the 0–255 scale. Input + residual = marked Pixel (x=904, y=993), red · zero-based Before truncation: 156.12 Drop the fraction: 155 + 1.12 = 156
Resolution scaling: our Boat example

How do the bits enter the image?

The preprocessing module E_pre enlarges the watermark representation to the cover image’s dimensions, concatenates their channels, and applies 3×3 convolution filters. This early fusion lets the main network process image content and message together. A convolution learns how to combine nearby feature values.

The main body adapts MUNIT, an image-translation network:

“Specifically, we remove channel normalization, double the network depth but halve the internal dimension, effectively reducing the model size 2x.”

— Bui et al. (2023), Section 3.1.1

Channel normalization rescales internal features. The embedder predicts the marked image directly; residual subtraction happens afterward for resolution scaling.

TrustMark embedder network in the released Q model: message path, E_pre, four downsampling stages, residual block, four upsampling stages with skip concatenations, E_post and encoded image, with stage shapes

Released Q model architecture; spatial shapes are height × width × channels.

In the released Q model, l=100 network bits enter a linear layer with 768 outputs, reshaped to 16×16×3. Nearest-neighbor interpolation and ReLU produce a 256×256×3 message map. Concatenating it with the 256×256×3 cover gives 256×256×6; E_pre applies a 3×3 convolution and SiLU to produce 256×256×32, so d=32.

The released Q body is Unet1: four 3×3 stride-2 ReLU convolutions expand channels from 32 to 512. One residual block uses two 3×3 convolutions, SiLU and input addition. Four decoder stages apply nearest upsampling, 2×2 SiLU convolution, skip concatenation and 3×3 SiLU convolution, ending at n=32. The final skip includes the fused input. Its 8,642,179 parameters include 77,568 for message projection and 1,760 for E_pre.

Why finish with several 1×1 convolutions?

E_post converts internal features into red, green and blue pixels. Each 1×1 convolution mixes channels at one location. SiLU activations supply nonlinear transformations between layers; a final tanh bounds output values to [-1, 1]. Section 3.1.1 motivates this module by preserving high-frequency detail such as edges.

Released Q E_post uses a leading 3×3 SiLU convolution at 32 channels, then 1×1 SiLU and tanh convolutions producing 16 and 3 channels. Their 9,248, 528 and 51 parameters total 9,827. Table 4 ablates E_post in TrustMark-B; GAN and FFL ablations remove training losses.

Figure 12: input, learned watermark, preprocessing activations, postprocessing activations and encoded output

Bui et al. (2023), Figure 12.

Supplement D.2 visualizes message and image content blended in the preprocessing activations, followed by postprocessing activations and the encoded image.

Table 4 changes components of TrustMark-B on DIV2K. With the other tested quality components retained, removing E_post changes PSNR from 40.203 to 38.366 dB and noised bit accuracy from 0.973 to 0.970. Bit accuracy is the fraction of bits predicted correctly. Section 4.5 reports that GAN, E_post and FFL individually add 0.7, 1.6 and 1.7 dB. E_post with either GAN or FFL adds about 3 dB; all three add about 4 dB PSNR and 1.4% bit accuracy. With all three disabled, the authors report PSNR equivalent to StegaStamp.

SettingPSNR (dB)Noised acc.
GAN + FFL + E_post40.2030.973
GAN + FFL, no E_post38.3660.970
FFL + E_post, no GAN39.6030.975
GAN + E_post, no FFL39.5260.974
E_post only38.9160.979
GAN only37.0340.963
FFL only38.0840.974
None of the three36.3000.959

Source: Bui et al. (2023), Table 4.

Why use ResNet-50 to recover the bits?

Section 3.1.2 replaces ResNet-50’s final layer with a sigmoid output for each bit. Sigmoid maps predictions into the interval between zero and one. Residual connections carry earlier features forward by addition.

TrustMark extractor network in the released Q model: two resizes, ResNet-50 stem and four residual stages, average pooling, FC to 100 logits, threshold and BCH_5 decoding, with stage shapes

Released Q extractor; spatial shapes are height × width × channels.

Released Q decoding bilinearly resizes to 245², normalizes to [-1, 1], then resizes to 224² in SecretDecoder. ResNet-50 applies a 7×7 stride-2 convolution, batch normalization, ReLU and 3×3 stride-2 max pooling, followed by four residual stages with 3, 4, 6 and 3 bottleneck blocks.

Average pooling and flattening feed a 2048→100 FC layer (204,900 parameters); the extractor totals 23,712,932. The paper describes sigmoid probabilities; released inference thresholds logits at zero, equivalent to sigmoid > 0.5. BCH_5 decodes the 100 predicted packet bits into 61 payload bits.

The DIV2K backbone ablation also converges with ResNet18, DenseNet121, RegNet and ResNeXt, but training fails with VGG, GoogLeNet, ConvNeXt and EfficientNet (Section 4.5). Every successful backbone has residual or skip connections. The authors attribute this requirement to the subtle watermark signal, complex noise simulation and staged training together.

BackbonePSNR (dB)Noised bit acc.
RegNet39.2590.986
ResNeXt39.4610.976
ResNet1838.8560.972
DenseNet12139.5690.982

Source: Bui et al. (2023), Table 4. Alternative extractor backbones that converged.

What does each loss protect?

Training combines recovery and quality penalties. The Section 4.1 weights give:

L_total = alpha * L_quality(x, y) + L_recovery(w, w_hat)
L_quality = 1.5 * L_YUV + 1 * L_LPIPS + 1.5 * L_FFL + 1 * L_GAN

Recovery uses binary cross-entropy, penalizing incorrect bit probabilities. YUV loss measures squared pixel differences in a color space separating brightness from color. LPIPS measures perceptual differences using learned features. GAN loss uses a discriminator trained to distinguish originals from encoded images, with a gradient penalty for stability.

“we propose to add a focal frequency loss (FFL) to bridge the gap between the cover and encoded image in frequency domain”

— Bui et al. (2023), Section 3.1.4

FFL dynamically weights differences between the images’ Fourier transforms, which represent spatial frequencies. Alpha controls the quality contribution.

Why unlock training in stages?

Supplement A.1 starts alpha at 0.05 to prioritize recovery. Training initially repeats a fixed cover batch with random watermarks; noise and GAN loss are off. At training-batch accuracy 0.90, random cover batches begin; at 0.95, noise begins; at 0.98, GAN training begins and alpha rises linearly to its maximum over 10,000 iterations. Enabled components remain active. Maximum alpha is 20 for TrustMark-B and 27.5 for TrustMark-Q. The authors find this schedule aids convergence with severe noise, large alpha or long messages.

StageTrigger acc.What turns on
0–Fixed cover batch; random watermarks
10.90Random cover batches
20.95Noise simulation
30.98GAN loss; alpha ramping

Source: Bui et al. (2023), Table 5. – means the initial stage; enabled components remain active.

Can the quality–recovery balance change after training?

Supplement C introduces a residual multiplier lambda at inference (Equation 7):

y = clamp(x + lambda * r, -1, 1)

Here x is the normalized cover and r is the resized residual. Lower lambda improves imperceptibility and lowers bit accuracy; higher lambda strengthens recovery at the cost of image quality. The authors identify [0.75, 1.50] as a reasonable range and allow the setting to change for each image. Figure 15 evaluates this control on DIV2K.

Figure 15: PSNR and bit-accuracy curves with the stated lambda range highlighted

Bui et al. (2023), Figure 15. The highlight spans lambda 0.75–1.50.

Alpha_max sets the trade-off during training. Section 4.2 reports stable training over (0, 30): increasing alpha_max to 30 improves PSNR by more than 8 dB while bit accuracy drops by less than 5% (Figure 6).

Figure 6: PSNR and bit accuracy against alpha_max, with the values at the largest alpha_max highlighted

Bui et al. (2023), Figure 6. Highlights mark PSNR and bit accuracy at the largest alpha_max.

All parts of the training graph

The cover and message pass through E_pre, the MUNIT body and E_post. Differentiable noise transforms the encoded image before extraction, allowing recovery errors to propagate back to the embedder. Quality losses compare the original and encoded images; the discriminator supplies the adversarial term. Resolution scaling constructs the full-sized output at inference.

TrustMark training graph Cover and message enter E_pre, then MUNIT and E_post form the encoded image. Differentiable noise feeds the ResNet-50 extractor. Recovery loss compares recovered and embedded messages. YUV, LPIPS, FFL and GAN losses compare encoded images with covers. At inference the resized residual is added to the original cover. TrustMark: the complete training graph Cover + message E_preMUNIT bodyE_postNoiseExtractor Early fusionImage features1×1 → RGBDifferentiableResNet-50 Recovered bits Recovery BCE Cover reference Quality losses on cover and encoded image YUV + LPIPS + FFL + GAN discriminator Inference at the original resolution Small output − small input → resize residual → add to original cover Training gradients reach the embedder through noise and extraction.
TrustMark training graph (open full size)

What experiments did the authors run?

The authors compare methods, isolate image changes, vary training noise, attack the decoder, remove and replace marks, and test message length, alternative noise settings and resolution.

Main comparison

Table 1 tests CLIC, DIV2K and MetFace with a random watermark per image. Models train on 101,000 MIRFlickR images. TrustMark-Q prioritizes quality; TrustMark-B balances quality and recovery. Baselines include RoSteALS, RivaGAN, SSL, StegaStamp and dwtDctSvd.

The noised condition applies three geometric transforms plus two perturbations randomly selected from fifteen. Supplement A.2 specifies flip probability 50%, resize scale 0.8–1.0 with aspect ratio 3/4–4/3, and a 244-pixel crop from 256. High-severity settings include JPEG quality down to 40, brightness factors 0.5–1.5 and Gaussian-noise standard deviation 0.08; Table 6 gives every setting. Perturbations include compression, color changes, blur, noise and posterization.

TrustMark-Q’s mean PSNR is 43.26, 42.39 and 45.34 dB for CLIC, DIV2K and MetFace. Clean bit accuracy is 1.00 throughout; noised accuracy is 0.95, 0.95 and 0.96. Section 4.6 states that no reported results use error correction.

DataPSNR (dB)Clean acc.Noised acc.
CLIC43.261.000.95
DIV2K42.391.000.95
MetFace45.341.000.96

Source: Bui et al. (2023), Table 1. TrustMark-Q; means only.

Single changes

Figure 5 tests TrustMark-Q and baselines on DIV2K, separating individual noise sources from the mixed condition. TrustMark-Q gives bit accuracy 0.95 for brightness, 0.90 for Gaussian noise and 0.95 for mixed changes.

Figure 5 Brightness, Gaussian Noise and Mixed groups with TrustMark-Q values highlighted and model legend retained

Bui et al. (2023), Figure 5. Enlarged groups retain the model legend.

Training-noise strength

Figure 4b trains and tests TrustMark-B on DIV2K with no noise simulation and low, medium and high settings. With simulation enabled, geometric settings stay fixed; optional perturbation strength changes. JPEG minimum quality is 70, 50 and 40; Gaussian-noise standard deviation is 0.02, 0.04 and 0.08 (Table 6). Section 4.3 reports PSNR falling from 53.2 dB without noise to 40.2 dB at high severity, while bit accuracy falls by 3%.

Figure 4b: quality and bit-accuracy endpoints across training-noise settings

Bui et al. (2023), Figure 4b. Highlights show both metrics’ endpoints.

Adversarial attack

Figure 4c attacks those TrustMark-B variants on DIV2K with I-FGSM, the iterative fast gradient sign method. It uses decoder gradients to adjust a perturbation, initially zero, at each iteration. Maximum strength is epsilon = 8/255; the stopping criterion is bit accuracy below 0.5 + epsilon/2. Every image from the model trained without noise crosses that threshold in fewer than 50 iterations. With high-severity training noise, 32% require more than 3000 iterations.

Figure 4c with fastest no-noise and slowest high-noise attack bins highlighted

Bui et al. (2023), Figure 4c. Bins count required attack iterations.

Removal and re-watermarking

Table 3 tests removal of TrustMark-B marks on DIV2K. TrustMark-RM uses KBNet to restore the cover. Training freezes the embedder and extractor, inserts three random watermarks per cover, and penalizes pixel differences, discriminator output and differences from the cover’s extractor response. Algorithm 1 transfers removal to full resolution. I-FGSM attacks extraction with the strength and stopping rule above.

The released trustmark_rm_Q remover is KBNet_s with 38,321,379 parameters. A 3×3 intro convolution produces 32 channels. Four encoder stages use 2/2/4/8 KB blocks and downsample to 512 channels; 12 middle KB blocks follow. Four decoders upsample with skip additions and 2 KB blocks each, returning to 32 channels. A 3×3 RGB convolution precedes input addition and wrapper clamping to [-1, 1].

TrustMark remover KBNet_s network: intro convolution, four encoder stages, middle blocks, four decoder stages with skip additions, RGB convolution, input addition and clamp, with stage shapes

Released trustmark_rm_Q remover; spatial shapes are height × width × channels.

TrustMark-RM gives 48.48 dB and bit accuracy 0.553; I-FGSM gives 23.48 dB and 0.629. PSNR measures restoration relative to the cover; accuracy nearer random guessing indicates less remaining message signal. Section 4.4 attributes both staying above 0.5 to resolution scaling weakening removal. Repeated removal and replacement on DIV2K preserves quality better than replacement alone, but unwanted changes accumulate and the remover’s denoising effect weakens across rounds. Bit accuracy is unaffected by whether removal is used.

MethodPSNR (dB)SSIMBit acc.
TrustMark-RM48.480.9970.553
I-FGSM23.480.6130.629

Source: Bui et al. (2023), Table 3.

Other tests and limits

Increasing TrustMark-B’s message from 32 to 200 bits on DIV2K lowers PSNR by 7.5 dB and accuracy by 11% (Figure 4a). Table 2 replaces the noise model with ImageNet-C for training and evaluation on CLIC, disables resolution scaling, and evaluates at the model’s designed resolution: TrustMark with maximum alpha 15 gives 38.87 dB and noised accuracy 0.95. Section 4.6 identifies cluttered images as difficult cases, hypothesizing interference between subtle watermarks and high-frequency content; PSNR and extraction accuracy fall slightly.

Our test: does the payload survive image changes at both sizes?

Setup

SettingValue
AirplaneColor; red, green, blue channels
BoatGrayscale repeated across three channels
Marking sizes512×512; 2048×2048
PayloadFixed 61 bits; BCH_5 correction
PreparationLanczos enlargement before marking
Single changesOriginal 7 changes; extended A: 17 settings, brightness 1.5 shared
Combined changesB: 20 seeded draws per image and size; geometry then two perturbations
RemovalC: trustmark_rm_Q; removal, re-watermarking and JPEG 50 removal
Decoder-gradient attackD: I-FGSM on each image and size
Strength sweepE: lambda 0.5, 0.75, 1.0, 1.25, 1.5; unchanged and JPEG 50

We tested with released Q encoder and decoder weights in TrustMark version 0.9.2. Our code and numeric results are on GitHub. The extended experiment code is in the same folder.

The inputs are USC-SIPI Airplane and Boat. Each enlarged input is prepared from the same native image before embedding the fixed payload.

Each marked image is decoded unchanged and after each separate change: JPEG quality 90, 70 or 50; downscaling both dimensions to 50%; Gaussian blur with radius 2 pixels; brightness multiplied by 1.5; and a center crop retaining 80% of each side, rounded to whole pixels. Downscaled copies go directly to decoding.

The panels show the native input, marked output and absolute difference. Differences are amplified 16 times and clipped at 255 for display. The second row shows changed copies and complete-payload recovery after correction.

Airplane input, marked output, amplified absolute difference and changed copies with exact-recovery labels
Boat input, marked output, amplified absolute difference and changed copies with exact-recovery labels

Quality

Embedding quality · our measurementsPSNR of the marked PNG against its input, four image/size pairs.Embedding qualityAirplaneBoat5122048Ring: exact recovery failedPSNR against input (dB)01020304050Airplane 512Airplane 512, none: 61/61; exact=True41.04Airplane 2048Airplane 2048, none: 61/61; exact=True40.97Boat 512Boat 512, none: 61/61; exact=True43.32Boat 2048Boat 2048, none: 61/61; exact=True43.26
Embedding quality · our measurements (open full size)

PSNR was 41.04 dB for Airplane and 43.32 dB for Boat at 512×512. At 2048×2048, the values were 40.97 and 43.26 dB, respectively.

Single changes

Single changes: raw bits and exact recovery · our measurements23 distinct settings: seven original and seventeen extended, brightness 1.5 deduplicated. Each row has four series; orange rings denote failed exact recovery after BCH.Single changes: raw bits and exact recoveryAirplaneBoat5122048Ring: exact recovery failedWithin each row: Airplane 512, 2048; Boat 512, 2048.Raw payload bits correct /614550556061JPEG 90Airplane 512, jpeg90: 61/61; exact=TrueAirplane 2048, jpeg90: 61/61; exact=TrueBoat 512, jpeg90: 61/61; exact=TrueBoat 2048, jpeg90: 60/61; exact=TrueJPEG 70Airplane 512, jpeg70: 61/61; exact=TrueAirplane 2048, jpeg70: 61/61; exact=TrueBoat 512, jpeg70: 60/61; exact=TrueBoat 2048, jpeg70: 60/61; exact=TrueJPEG 50Airplane 512, jpeg50: 61/61; exact=TrueAirplane 2048, jpeg50: 61/61; exact=TrueBoat 512, jpeg50: 58/61; exact=TrueBoat 2048, jpeg50: 59/61; exact=TrueDownscale 50%Airplane 512, downscale50: 61/61; exact=TrueAirplane 2048, downscale50: 61/61; exact=TrueBoat 512, downscale50: 61/61; exact=TrueBoat 2048, downscale50: 61/61; exact=TrueGaussian blur, r=2Airplane 512, blur: 61/61; exact=TrueAirplane 2048, blur: 61/61; exact=TrueBoat 512, blur: 61/61; exact=TrueBoat 2048, blur: 61/61; exact=TrueBrightness 1.5Airplane 512, brightness: 50/61; exact=FalseAirplane 2048, brightness: 50/61; exact=FalseBoat 512, brightness: 61/61; exact=TrueBoat 2048, brightness: 61/61; exact=TrueCenter crop 80%Airplane 512, crop80: 61/61; exact=TrueAirplane 2048, crop80: 61/61; exact=TrueBoat 512, crop80: 61/61; exact=TrueBoat 2048, crop80: 61/61; exact=TrueGaussian noise, std 0.08Airplane 512, noise008: 60/61; exact=TrueAirplane 2048, noise008: 61/61; exact=TrueBoat 512, noise008: 57/61; exact=FalseBoat 2048, noise008: 60/61; exact=TrueContrast 1.5Airplane 512, contrast150: 61/61; exact=TrueAirplane 2048, contrast150: 61/61; exact=TrueBoat 512, contrast150: 61/61; exact=TrueBoat 2048, contrast150: 61/61; exact=TrueContrast 0.5Airplane 512, contrast050: 61/61; exact=TrueAirplane 2048, contrast050: 61/61; exact=TrueBoat 512, contrast050: 61/61; exact=TrueBoat 2048, contrast050: 61/61; exact=TrueSaturation 0.5Airplane 512, saturation050: 61/61; exact=TrueAirplane 2048, saturation050: 61/61; exact=TrueBoat 512, saturation050: 61/61; exact=TrueBoat 2048, saturation050: 61/61; exact=TrueSaturation 1.5Airplane 512, saturation150: 61/61; exact=TrueAirplane 2048, saturation150: 61/61; exact=TrueBoat 512, saturation150: 61/61; exact=TrueBoat 2048, saturation150: 61/61; exact=TrueHue +0.05 turnAirplane 512, hue005: 61/61; exact=TrueAirplane 2048, hue005: 61/61; exact=TrueBoat 512, hue005: 61/61; exact=TrueBoat 2048, hue005: 61/61; exact=TrueSharpness 2.5Airplane 512, sharpness250: 61/61; exact=TrueAirplane 2048, sharpness250: 61/61; exact=TrueBoat 512, sharpness250: 61/61; exact=TrueBoat 2048, sharpness250: 61/61; exact=TrueMedian blur, k=3Airplane 512, median3: 61/61; exact=TrueAirplane 2048, median3: 61/61; exact=TrueBoat 512, median3: 61/61; exact=TrueBoat 2048, median3: 61/61; exact=TrueBox blur, k=7Airplane 512, box7: 61/61; exact=TrueAirplane 2048, box7: 61/61; exact=TrueBoat 512, box7: 61/61; exact=TrueBoat 2048, box7: 61/61; exact=TrueMotion blur, k=9, 45°Airplane 512, motion9_45: 61/61; exact=TrueAirplane 2048, motion9_45: 61/61; exact=TrueBoat 512, motion9_45: 61/61; exact=TrueBoat 2048, motion9_45: 61/61; exact=TruePosterize 3 bitsAirplane 512, posterize3: 55/61; exact=FalseAirplane 2048, posterize3: 56/61; exact=FalseBoat 512, posterize3: 61/61; exact=TrueBoat 2048, posterize3: 61/61; exact=TrueRGB shift ±0.1Airplane 512, rgb010: 61/61; exact=TrueAirplane 2048, rgb010: 61/61; exact=TrueBoat 512, rgb010: 61/61; exact=TrueBoat 2048, rgb010: 61/61; exact=TrueGrayscaleAirplane 512, grayscale: 58/61; exact=TrueAirplane 2048, grayscale: 58/61; exact=TrueBoat 512, grayscale: 58/61; exact=TrueBoat 2048, grayscale: 58/61; exact=TrueHorizontal flipAirplane 512, hflip: 61/61; exact=TrueAirplane 2048, hflip: 61/61; exact=TrueBoat 512, hflip: 61/61; exact=TrueBoat 2048, hflip: 61/61; exact=TrueBrightness 0.5Airplane 512, brightness050: 61/61; exact=TrueAirplane 2048, brightness050: 61/61; exact=TrueBoat 512, brightness050: 61/61; exact=TrueBoat 2048, brightness050: 61/61; exact=TrueJPEG 40Airplane 512, jpeg40: 61/61; exact=TrueAirplane 2048, jpeg40: 61/61; exact=TrueBoat 512, jpeg40: 58/61; exact=TrueBoat 2048, jpeg40: 59/61; exact=True
Single changes: raw bits and exact recovery · our measurements (open full size)

The chart counts raw payload bits out of 61; an orange ring marks failed exact recovery after BCH. Airplane failed exact recovery after posterization to 3 bits and brightness 1.5 at both sizes; Boat failed after Gaussian noise at 512×512. Grayscale gave 58/61 raw bits and exact recovery in every image/size pair. Unmarked inputs returned detected=false and an empty payload at both sizes.

Combined changes

Combined changes: twenty draws per pair · our measurementsOne dot per saved draw, raw payload bits on the shared horizontal axis. The orange ring records failure of exact BCH recovery, independently of raw bit count.Combined changes: twenty draws per pairAirplaneBoat5122048Ring: exact recovery failedEach strip: draws 00–19, ordered from top to bottom.Raw payload bits correct /614550556061Airplane 512Airplane 512, draw00: 61/61; exact=True00Airplane 512, draw01: 61/61; exact=TrueAirplane 512, draw02: 61/61; exact=TrueAirplane 512, draw03: 61/61; exact=TrueAirplane 512, draw04: 61/61; exact=TrueAirplane 512, draw05: 61/61; exact=True05Airplane 512, draw06: 61/61; exact=TrueAirplane 512, draw07: 60/61; exact=TrueAirplane 512, draw08: 61/61; exact=TrueAirplane 512, draw09: 61/61; exact=TrueAirplane 512, draw10: 61/61; exact=True10Airplane 512, draw11: 61/61; exact=TrueAirplane 512, draw12: 61/61; exact=TrueAirplane 512, draw13: 60/61; exact=FalseAirplane 512, draw14: 61/61; exact=TrueAirplane 512, draw15: 59/61; exact=False15Airplane 512, draw16: 61/61; exact=TrueAirplane 512, draw17: 61/61; exact=TrueAirplane 512, draw18: 61/61; exact=TrueAirplane 512, draw19: 61/61; exact=True19Exact 18/20; mean 60.80/61; min 59/61Airplane 2048Airplane 2048, draw00: 61/61; exact=True00Airplane 2048, draw01: 61/61; exact=TrueAirplane 2048, draw02: 60/61; exact=TrueAirplane 2048, draw03: 61/61; exact=TrueAirplane 2048, draw04: 61/61; exact=TrueAirplane 2048, draw05: 61/61; exact=True05Airplane 2048, draw06: 61/61; exact=TrueAirplane 2048, draw07: 61/61; exact=TrueAirplane 2048, draw08: 61/61; exact=TrueAirplane 2048, draw09: 61/61; exact=TrueAirplane 2048, draw10: 61/61; exact=True10Airplane 2048, draw11: 61/61; exact=TrueAirplane 2048, draw12: 61/61; exact=TrueAirplane 2048, draw13: 61/61; exact=TrueAirplane 2048, draw14: 57/61; exact=TrueAirplane 2048, draw15: 61/61; exact=True15Airplane 2048, draw16: 61/61; exact=TrueAirplane 2048, draw17: 61/61; exact=TrueAirplane 2048, draw18: 56/61; exact=FalseAirplane 2048, draw19: 61/61; exact=True19Exact 19/20; mean 60.50/61; min 56/61Boat 512Boat 512, draw00: 61/61; exact=True00Boat 512, draw01: 61/61; exact=TrueBoat 512, draw02: 59/61; exact=TrueBoat 512, draw03: 58/61; exact=TrueBoat 512, draw04: 57/61; exact=FalseBoat 512, draw05: 55/61; exact=False05Boat 512, draw06: 60/61; exact=TrueBoat 512, draw07: 54/61; exact=FalseBoat 512, draw08: 49/61; exact=FalseBoat 512, draw09: 61/61; exact=TrueBoat 512, draw10: 61/61; exact=True10Boat 512, draw11: 52/61; exact=FalseBoat 512, draw12: 46/61; exact=FalseBoat 512, draw13: 58/61; exact=TrueBoat 512, draw14: 61/61; exact=TrueBoat 512, draw15: 61/61; exact=True15Boat 512, draw16: 59/61; exact=TrueBoat 512, draw17: 57/61; exact=TrueBoat 512, draw18: 60/61; exact=TrueBoat 512, draw19: 61/61; exact=True19Exact 14/20; mean 57.55/61; min 46/61Boat 2048Boat 2048, draw00: 61/61; exact=True00Boat 2048, draw01: 61/61; exact=TrueBoat 2048, draw02: 61/61; exact=TrueBoat 2048, draw03: 61/61; exact=TrueBoat 2048, draw04: 61/61; exact=TrueBoat 2048, draw05: 61/61; exact=True05Boat 2048, draw06: 55/61; exact=FalseBoat 2048, draw07: 57/61; exact=TrueBoat 2048, draw08: 61/61; exact=TrueBoat 2048, draw09: 60/61; exact=TrueBoat 2048, draw10: 59/61; exact=True10Boat 2048, draw11: 61/61; exact=TrueBoat 2048, draw12: 61/61; exact=TrueBoat 2048, draw13: 61/61; exact=TrueBoat 2048, draw14: 61/61; exact=TrueBoat 2048, draw15: 61/61; exact=True15Boat 2048, draw16: 61/61; exact=TrueBoat 2048, draw17: 61/61; exact=TrueBoat 2048, draw18: 61/61; exact=TrueBoat 2048, draw19: 60/61; exact=True19Exact 19/20; mean 60.30/61; min 55/61
Combined changes: twenty draws per pair · our measurements (open full size)

Across 20 draws, exact recovery was 18/20 and 19/20 for Airplane at 512×512 and 2048×2048; mean raw recovery was 60.80/61 and 60.50/61, with minima 59/61 and 56/61. Boat gave 14/20 and 19/20 exact, means 57.55/61 and 60.30/61, and minima 46/61 and 55/61.

Decoder-gradient attack (I-FGSM)

The attack used epsilon 8/255, step 1/255, a stopping threshold below 0.5 + epsilon/2 on the 100 network bits, and a cap of 500 steps.

ImageSizeStepsBits after PNG (/61)
Airplane512233
Airplane2048233
Boat512427
Boat2048229

PSNR against the marked image was 44.29 and 44.30 dB for Airplane at 512 and 2048, and 42.11 and 45.15 dB for Boat. All four saved PNGs returned detected=false.

Airplane and Boat at 512: marked image, attacked PNG preview and absolute difference amplified 16 times

Removal and re-watermarking

At strength 1.0, we applied the Q remover to each marked image, to each unmarked input, and to each JPEG 50 marked copy; re-watermarking used the bitwise complement of the fixed 61-bit payload.

ImageSizePSNR vs inputBits /61 after removal
Airplane51240.5839
Airplane204840.5240
Boat51243.3645
Boat204843.2744

Removal did not recover the original payload exactly in any of the four cases. Airplane at 512 returned detected=true with a different-length payload; the other three returned detected=false and an empty payload.

For removal on unmarked inputs, PSNR against input was 35.76 and 37.64 dB for Airplane at 512 and 2048, and 47.31 and 47.01 dB for Boat; all returned detected=false and an empty payload.

Both direct re-watermarking and removal followed by re-watermarking recovered the second payload exactly in every image/size pair: 61/61 raw bits matched the second payload, zero matched the first, and detected=true throughout. PSNR against input for the direct and removal-first paths was 38.47/38.33 and 38.40/38.24 dB for Airplane at 512 and 2048, and 40.31/40.27 and 40.25/40.18 dB for Boat.

After JPEG 50 and removal, raw recovery was 49/61 and 36/61 for Airplane at 512 and 2048, and 48/61 and 46/61 for Boat, with detected=false and no exact recovery throughout.

Strength sweep

Residual strength: quality and exact recovery · our measurementsPSNR is measured before JPEG. Lines use actual lambda. Size markers are displaced fourteen SVG units sideways and recovery-condition markers six units to either side; left is unchanged, right is JPEG 50. Orange rings mark failed exact recovery.Residual strength: quality and exact recoveryAirplaneBoat5122048Ring: exact recovery failedLeft marker: unchanged. Right marker: JPEG 50.512 circles and 2048 squares are offset sideways for visibility.PSNR against input (dB)363840424446480.50.7511.251.5Residual multiplier lambdaAirplane 512, lambda0.5-unchanged: 56/61; exact=FalseAirplane 512, lambda0.5-jpeg50: 52/61; exact=FalseAirplane 512, lambda0.75-unchanged: 61/61; exact=TrueAirplane 512, lambda0.75-jpeg50: 60/61; exact=TrueAirplane 512, lambda1.0-unchanged: 61/61; exact=TrueAirplane 512, lambda1.0-jpeg50: 61/61; exact=TrueAirplane 512, lambda1.25-unchanged: 61/61; exact=TrueAirplane 512, lambda1.25-jpeg50: 61/61; exact=TrueAirplane 512, lambda1.5-unchanged: 61/61; exact=TrueAirplane 512, lambda1.5-jpeg50: 61/61; exact=TrueAirplane 2048, lambda0.5-unchanged: 56/61; exact=FalseAirplane 2048, lambda0.5-jpeg50: 55/61; exact=FalseAirplane 2048, lambda0.75-unchanged: 61/61; exact=TrueAirplane 2048, lambda0.75-jpeg50: 61/61; exact=TrueAirplane 2048, lambda1.0-unchanged: 61/61; exact=TrueAirplane 2048, lambda1.0-jpeg50: 61/61; exact=TrueAirplane 2048, lambda1.25-unchanged: 61/61; exact=TrueAirplane 2048, lambda1.25-jpeg50: 61/61; exact=TrueAirplane 2048, lambda1.5-unchanged: 61/61; exact=TrueAirplane 2048, lambda1.5-jpeg50: 61/61; exact=TrueBoat 512, lambda0.5-unchanged: 58/61; exact=TrueBoat 512, lambda0.5-jpeg50: 36/61; exact=FalseBoat 512, lambda0.75-unchanged: 61/61; exact=TrueBoat 512, lambda0.75-jpeg50: 58/61; exact=TrueBoat 512, lambda1.0-unchanged: 61/61; exact=TrueBoat 512, lambda1.0-jpeg50: 58/61; exact=TrueBoat 512, lambda1.25-unchanged: 61/61; exact=TrueBoat 512, lambda1.25-jpeg50: 59/61; exact=TrueBoat 512, lambda1.5-unchanged: 61/61; exact=TrueBoat 512, lambda1.5-jpeg50: 61/61; exact=TrueBoat 2048, lambda0.5-unchanged: 58/61; exact=TrueBoat 2048, lambda0.5-jpeg50: 35/61; exact=FalseBoat 2048, lambda0.75-unchanged: 61/61; exact=TrueBoat 2048, lambda0.75-jpeg50: 56/61; exact=FalseBoat 2048, lambda1.0-unchanged: 61/61; exact=TrueBoat 2048, lambda1.0-jpeg50: 59/61; exact=TrueBoat 2048, lambda1.25-unchanged: 61/61; exact=TrueBoat 2048, lambda1.25-jpeg50: 59/61; exact=TrueBoat 2048, lambda1.5-unchanged: 61/61; exact=TrueBoat 2048, lambda1.5-jpeg50: 61/61; exact=TrueAirplaneBoat
Residual strength: quality and exact recovery · our measurements (open full size)

The curves show embedding PSNR at lambda 0.5–1.5. Each point has separate exact-recovery markers for the unchanged image and JPEG 50. At 512×512, PSNR fell from 46.21 to 37.69 dB for Airplane and from 47.81 to 40.12 dB for Boat as lambda rose from 0.5 to 1.5. At lambda 0.5, Airplane failed exact recovery even unchanged, with 56/61 raw bits at both sizes, and Boat failed after JPEG 50 with 36/61 and 35/61. At lambda 0.75, only Boat at 2048×2048 failed after JPEG 50, with 56/61. Every case from lambda 1.0 upward recovered exactly.

Conditions and limits

The decoder needs compatible trained weights and the encoding format. It operates without an original reference. A recovered identifier needs an associated record to supply origin information; its recovery alone cannot verify that record’s truth.

The measured scope covers the source images and fixed payload used for separate changes, combined changes, decoder-gradient attacks, removal and a residual-strength sweep, plus its bitwise complement for re-watermarking. Enlarged inputs contain interpolated source pixels; broader payload variation and extreme aspect ratios remain untested.

What came after

Adobe’s TrustMark repository states that the work was later published at ICCV 2025 as TrustMark: Robust Watermarking and Watermark Removal for Arbitrary Resolution Images.

Implementation details

Network sizes in the released Q model, with batch dimension omitted:

ModuleParametersInput → output shape
Embedder8,642,179256×256×3 + 100 bits → 256×256×3
Extractor23,712,932245×245×3 → 224×224×3 → 100 logits
TrustMark-RM38,321,379256×256×3 → 256×256×3

Source: released Q configuration, module parameters and forward-hook shapes in experiment4.

SettingValue
Package and weightsTrustMark 0.9.2; Q
Removertrustmark_rm_Q; official MD5: yaml 8476bcd4092abf302272868f3b4c2e38; ckpt 760337a5596e665aed2ab5c49aa5284f
PacketBCH_5: 61 payload + 35 parity + 4 schema bits
Encoder / decoder input256×256 / 245×245
Residual strength1.0; WM_STRENGTH varied in the sweep
Seed20231130; per-setting and per-draw offsets
InterpolationLanczos input enlargement; bilinear residual merge
Attack resizeBilinear, align_corners=False, antialias=True

BCH adds redundant parity bits to repair errors; raw payload counts precede correction, and exact recovery requires the corrected payload to equal the embedded message. The released encoder adjusts each residual channel’s average and blends changes near edges. BCH parity-handling and validity-check fixes are included with our code.

The perturbations use PIL/NumPy analogues of the paper’s Kornia operations. Float operations clip to [0, 1] and round to pixel integers. Contrast blends with mean luminance; saturation blends with grayscale; sharpness extrapolates from PIL’s smoothed image. Hue uses an integer HSV-channel shift. These definitions and rounding differ from Kornia.

SettingOursPaper (high)
Gaussian noisex + N(0, 0.08²)std. 0.08
Contrast / brightness / saturationFactors 0.5 and 1.50.5–1.5 each
Hue+13/256 turn; round(0.05×256)0.05
SharpnessFactor 2.52.5
Median / box blurMedian 3×3; box mean 7×7k=3 / k=7
Motion blurMean of x[i+d,j+d], d=−4…4; replicated edges; 45°, direction 0k=3–9; angle −90…90; direction −1…1
Posterizex & 224; retain 3 bits3 bits
RGB shiftx + (0.1, −0.1, 0.1)shift limit 0.1
GrayscaleForced luminance conversionp=0.5
JPEGQuality 40; 4:2:0minimum quality 40

Source: Bui et al. (2023), Table 6.

Combined draws mirror with probability 0.5, sample linear scale s uniformly over [0.8, 1.0] and aspect a log-uniformly over [3/4, 4/3], then resize to round(Ws√a) by round(Hs/√a). A random crop retains 244/256 of each resized side; two distinct perturbation types follow in sampled order. Single-change horizontal mirroring is forced. Supplement A.2 specifies the flip probability, scale/aspect ranges and crop ratio; the sampling and dimension formulas define our implementation.

References

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *