From Open Loop to Closed Loop: A Test-Time Iterative Optimization Framework for Reference-Consistent Image Generation

While controllable image generation has made significantstrides by incorporating visual reference conditions, existing methodspredominantly operate as open-loop systems. They inject control sig-nals in a strictly feed-forward manner, failing to guarantee strict fidelityto the reference due to the absence of active feedback and error cor-rection mechanisms. To address this fundamental limitation, we pro-pose a novel test-time iterative optimization framework that reformu-lates reference-consistent generation as a closed-loop dynamic trackingproblem. By treating the pre-trained generative model as a control plant,our framework employs a sensor-controller architecture driven by a mod-ified Proportional-Integral-Derivative (PID) algorithm. This mechanismiteratively optimizes the latent control signals at test time based on thesensed discrepancy between the generated output and the reference tar-get. Notably, this approach is entirely training-free, model-agnostic, andintegrates seamlessly around existing diffusion pipelines. Extensive eval-uations across ID-preserving, pose-controlled, and depth-controlled gen-eration tasks validate the universality of our method. Empirical resultsdemonstrate improvements over computation-matched open-loop base-lines, achieving relative performance gains of up to 25.36% for facialsimilarity, alongside spatial error reductions of up to 27.71% for posealignment and 28.50% for depth consistency. More broadly, this workoffers a new conceptual perspective: it demonstrates that controllablegeneration can be effectively managed as a dynamic feedback system,bringing the rigorous principles of classical control theory into the op-timization of generative models. Code is available at https://github.com/zzdrill/From-Open-Loop-to-Closed-Loop.