EgoSim: Egocentric World Simulator for Embodiment Interaction Generation

We introduce EgoSim, a closed-loop egocentric world sim-ulator that generates spatially consistent interaction videos and per-sistently updates the underlying 3D scene state for continuous simu-lation. Existing egocentric simulators either lack explicit 3D ground-ing, causing structural drift under viewpoint changes, or treat the sceneas static, failing to update world states across multi-stage interactions.EgoSim addresses both limitations by modeling 3D scenes as updatableworld states. We generate embodiment interactions via a Geometry-action-aware Observation Simulation model, with spatial consistencyfrom an Interaction-aware State Updating module. To overcome the crit-ical data bottleneck posed by the difficulty in acquiring densely alignedscene–interaction training pairs, we design a scalable pipeline that ex-tracts static point clouds, camera trajectories, and embodiment actionsfrom in-the-wild large-scale monocular egocentric videos. Extensive ex-periments demonstrate that EgoSim significantly outperforms existingmethods in terms of visual quality, spatial consistency, and generalizationto complex scenes and in-the-wild dexterous interactions, while support-ing cross-embodiment transfer to robotic manipulation. Project page isat: egosimulator.github.io.