HO-Flow: Generalizable Hand-Object Interaction Generation with Latent Flow Matching

Generating realistic 3D hand-object interactions (HOI) is afundamental challenge in computer vision and robotics, requiring bothtemporal coherence and high-fidelity physical plausibility. Existing meth-ods remain limited in their ability to learn expressive motion repre-sentations for generation and perform temporal reasoning. In this pa-per, we present HO-Flow, a framework for synthesizing realistic hand-object motion sequences from texts and canonical 3D objects. HO-Flowfirst employs an interaction-aware variational autoencoder to encode se-quences of hand and object motions into a unified latent manifold byincorporating hand and object kinematics, enabling the representationto capture rich interaction dynamics. It then leverages a masked flowmatching model that combines auto-regressive temporal reasoning withcontinuous latent generation, improving temporal coherence. To furtherenhance generalization, HO-Flow predicts object motions relative to theinitial frame, enabling effective pre-training on large-scale synthetic data.Experiments on the GRAB, OakInk, and DexYCB benchmarks demon-strate that HO-Flow achieves state-of-the-art performance in both phys-ical plausibility and motion diversity for interaction motion synthesis.