ComplexMimic: Human–Scene Interaction Imitation in Complex 3D Environments

Physics-based Human-Scene Interaction (HSI) imitation learn-ing is crucial for embodied intelligence as it bridges the gap betweenkinematic 3D motions and real-world dynamics. However, most exist-ing methods focus on simplified scene settings, leaving complex envi-ronments largely unexplored, which limits their applicability in real-world scenarios. In this paper, we focus on HSI mimicry in complexenvironments. Under this complex setting, we observe an inherent trade-off between successfully performing interaction and maintaining natu-ral, physically plausible motions. To address this challenge, we proposeComplexMimic, a framework that reconstructs diverse HSI by inter-preting imperfect MoCap data. First, we introduce a Dual Flow Strat-egy, which learns two complementary experts: an imitation expert foraccurate motion tracking and an interaction expert for collision-awareadaptation in complex scenes. Second, naive multi-expert distillation,which treats all experts equally, often under-samples challenging be-haviors, limiting effective learning. To mitigate this issue, we proposea difficulty-aware distillation strategy that adaptively weights super-vision and prioritizes hard-yet-learnable trajectories guided by failurestatistics and learning progress signals. Extensive experiments on threebenchmark datasets demonstrate that our approach outperforms cur-rent state-of-the-art methods. Our implementation is available at https://github.com/LuPan23/ComplexMimic.