Achieving Subcategorical Erasure in Text-to-Image Models

The emergence of large-scale text-to-image diffusion (T2ID)models has led to significant advancements in generating high-qualityvisual content from textual prompts. However, these powerful capabil-ities have also raised growing concerns about the generation of harm-ful and copyrighted material. While existing concept erasure techniquescan effectively block the production of specific unwanted concepts fromprompts, they often fall short when it comes to erasing an entire cate-gory (including subcategories) and are typically limited to handling onlya few concepts at a time. In this paper, we introduce SubcategoricalUnlearning via Regularized Erasure (SURE), a novel method for re-moving entire subcategories from text-to-image diffusion models usingonly a single parent category as the target. Unlike prior approaches,SURE does not rely on sets of synonyms. Instead, it employs conceptspace to discover and eliminate the target category while preserving themodel’s overall utility. To further enhance erasure, SURE incorporatesLipschitz regularization, which encourages smoother model responses toperturbations around the target category. Specifically, the regularizationpromotes consistent behavior in the model’s latent space when exposedto slight variations of the category to be forgotten. This smoothnessconstraint aids in erasure while maintaining the model’s ability to gen-erate unrelated content. Extensive experiments conducted across threetasks—object removal, suppression of explicit content, and elimination ofartistic styles demonstrate that SURE achieves balanced performance inboth effective category erasure and preservation of non-target concepts.