Learning to Stylize by Learning to Destylize: A Scalable Paradigm for Supervised Style Transfer

This paper introduces a scalable paradigm for supervisedstyle transfer by inverting the problem: instead of learning to stylizedirectly, we learn to destylize, reducing stylistic elements from artisticimages to recover their natural counterparts and thereby producing au-thentic, pixel-aligned training pairs at scale. To realize this paradigm, wepropose DeStylePipe, a progressive, multi-stage destylization frameworkthat begins with global general destylization, advances to category-wiseinstruction adaptation, and ultimately deploys specialized model adap-tation for complex styles that prompt engineering alone cannot han-dle. Tightly integrated into this pipeline, DestyleCoT-Filter employsChain-of-Thought reasoning to assess content preservation and style re-moval at each stage, routing challenging samples forward while discard-ing persistently low-quality pairs. Built on this framework, we constructDeStyle-350K, a large-scale dataset aligning diverse artistic styles withtheir underlying content. We further introduce BCS-Bench, a benchmarkfeaturing balanced content generality and style diversity for systematicevaluation. Extensive experiments demonstrate that models trained onDeStyle-350K achieve superior stylization quality, validating destyliza-tion as a reliable and scalable supervision paradigm for style transfer. Ourproject page: https://wangyephd.github.io/projects/DeStyle/index.html