Interact3D: Compositional 3D Generation of Interactive Objects

Recent breakthroughs in 3D generation have enabled thesynthesis of high-fidelity individual assets. However, generating 3D com-positional objects from single images–particularly under occlusions–remainschallenging. Existing methods often degrade geometric details in hiddenregions and fail to preserve the underlying object-object spatial rela-tionships (OOR). We present a novel framework Interact3D designedto generate physically plausible interacting 3D compositional objects.Our approach first leverages advanced generative priors to curate high-quality individual assets with a unified 3D guidance scene. To physicallycompose these assets, we then introduce a robust two-stage compositionpipeline. Based on the 3D guidance scene, the primary object is anchoredthrough precise global-to-local geometric alignment (registration), whilesubsequent geometries are integrated using a differentiable Signed Dis-tance Field (SDF)-based optimization that explicitly penalizes geometryintersections. To reduce challenging collisions, we further deploy a closed-loop, agentic refinement strategy. A Vision-Language Model (VLM) au-tonomously analyzes multi-view renderings of the composed scene, for-mulates targeted corrective prompts, and guides an image editing mod-ule to iteratively self-correct the generation pipeline. Extensive exper-iments demonstrate that Interact3D successfully produces promisingcollision-aware compositions with improved geometric fidelity and con-sistent spatial relationships. The code and dataset will be available athttps://github.com/SII-Hui/Interact3D.