Through Van Gogh’s Eyes: Global Style Transfer with Diffusion Model

Artistic image synthesis aims to recreate the expressive vi-sual identity of a target artist, yet existing methods often fail to cap-ture an artist’s global style. Conventional style transfer methods trans-fer the style of one or a few reference artworks to a content image ina One-to-One manner, making them effective for artwork-level styliza-tion but limited in representing the broader stylistic distribution of anartist. Text-to-image diffusion models conditioned on artist names, suchas ‘∼ in Van Gogh style’, offer greater flexibility, but they often sufferfrom text-induced bias and reproduce patterns from only a few iconicworks. To address these limitations, we introduce Global Style Transfer(GST), an artistic image synthesis paradigm, in a Many-to-One man-ner,that aggregates multiple artworks from a target artist and transferstheir shared global style to a single content image. For GST, we proposeGlobal Style Guidance (GSG), which learns a residual global style offset∆⟨t in the intermediate feature space, or h-space, of a diffusion modelunder a fixed prompt. By learning artist-level style semantics purely fromvisual statistics, GSG mitigates text-dependent artistic bias. We furtherpropose Content Alignment Guidance (CAG), a training-free perceptualguidance mechanism that preserves the semantic structure of the contentimage while allowing artist-specific geometric deformation. Experimentson WikiArt demonstrate that GST achieves superior stylistic fidelity,content preservation, and output diversity compared to existing styletransfer and diffusion-based artistic synthesis methods.