PIAvatar: Physically Interactive Avatars via Deformation Gradient Decoupling

3D human avatars have shown impressive visual fidelitydriven by pose-conditioned models, yet they still lack the physical abil-ity required for interactions with each other and environments. Althoughrecent studies have made various attempts to incorporate physical char-acteristics into 3D avatars, they only exhibit limited physical deforma-tions, often leading to constrained interaction behaviors. To resolve thisissue, we present PIAvatar, a framework to simultaneously enable physi-cally aware interactions between avatar-avatar and avatar-environment,and a non-rigid deformable human body simulation. In this work, ourkey insight is to decouple kinematic velocity from deformation gradi-ent. When external forces act on avatars, the kinematic velocity inducesstress which hinders the avatar’s ability to achieve a desired pose. Inaddition, we integrate a skeletal framework within the avatar. It allowsestimating its poses and real-time tracking in a closed form, even duringnon-rigid physical interactions. Our approach is implemented within aconventional Material Point Method framework to ensure physically con-sistent dynamics. We lastly evaluate the method on both human-objectand human-human interaction scenarios to assess its behavior under di-verse interaction settings.