FedOT: Ownership Verification and Leakage Tracing via Watermarks for Federated LDMs

Training Latent Diffusion Models (LDMs) within FederatedLearning (FL) has attracted increasing attention due to its ability tocombine the powerful generative capacity of LDMs with the privacy-preserving properties of FL. However, FL requires sharing the globalmodel with multiple participants, which risks unauthorized model dis-tribution or resale by malicious clients. While an intuitive approach isto adopt existing VAE-based watermarking techniques for LDMs in FL,this strategy falls short in addressing such threats due to two fundamen-tal challenges: (1) Existing methods support ownership verification butlack the ability to trace model leakage to a specific malicious client; (2)VAE-based watermarks are vulnerable, as they can be removed simplyby replacing the decoder with a clean counterpart. In this paper, we pro-pose FedOT, the first framework for ownership verification and leakagetracing in federated LDMs. Specifically, to address the first challenge, wedesign a chunked watermark, where the first part is for ownership verifi-cation, and the second part is used for client identification. Furthermore,to overcome the second challenge and secure the model against VAEreplacement attack, we introduce Latent Vector Transformation (LVT),which strengthens the connection between the VAE and U-Net latentspaces by modifying the original latent distribution of the VAE. Conse-quently, any attempt to replace the VAE for watermark removal leads tosignificant image quality degradation, making the LDM model unusable.Extensive experiments demonstrate that FedOT achieves superior per-formance in both ownership verification and traceability. Project page:https://spyzixuan.github.io/FedOT/.