XSemanticFlow: Cross Object Semantic Alignment for Zero-shot Manipulation

Establishing reliable correspondences across diverse objectinstances is fundamental to robust 3D understanding and generalizablemanipulation. However, existing methods are often inconsistent acrossgeometries or vulnerable to non-canonicalized poses, making correspon-dence transfer unreliable. To address this, we propose XSemanticFlow,which learns correspondences from pure semantic features. Given per-object semantic feature x001Celds, XSemanticFlow applies hierarchical intra-object self-attention and inter-object cross-attention to predict soft cor-respondence maps. By restricting cross-attention strictly to semantic to-kens, XSemanticFlow avoids geometric overx001Ctting, enabling it to predictgeneralizable correspondences across dix001Berent topologies and producealigned feature x001Celds. This enables XSemanticFlow to learn ex001Bectivelythrough self-supervised cross-pose alignment on large-scale unlabeledshapes, followed by supervised cross-instance x001Cnetuning on a cosegmen-tation dataset containing labeled object pairs from PartNet. We eval-uate our method on both 3D understanding and manipulation tasks.For 3D understanding, XSemanticFlow improves self-segmentation withSE(3) augmentations by +19.1 mIoU and +20.6 accuracy, and cross-instance cosegmentation by +10.9 mIoU and +12.5 accuracy over base-lines, demonstrating stronger transform consistency and semantic align-ment. For manipulation, XSemanticFlow provides a reliable alternativeto recent video-generation-based pipelines for zero-shot manipulation.Rather than relying on synthesized visual plans, we directly transfer con-tact regions and end-ex001Bector trajectories from a single reference demon-stration to novel instances, bypassing the brittle video generation failurescommon in contact-critical tasks.