Tac2Real: Reliable and GPU Visuotactile Simulation for Online Reinforcement Learning and Zero-shot Real-World Deployment

Visuotactile sensors are indispensable for contact-rich roboticmanipulation tasks. However, policy learning with tactile feedback insimulation, especially for online reinforcement learning (RL), remains acritical challenge, as it demands a delicate balance between physics fi-delity and computational efficiency. To address this challenge, we presentTac2Real, a lightweight visuotactile simulation framework designed toenable efficient online RL training. Tac2Real integrates the Precondi-tioned Nonlinear Conjugate Gradient Incremental Potential Contact (PNCG-IPC) method with a multi-node, multi-GPU high-throughput par-allel simulation architecture, which can generate marker displacementfields at interactive rates. Meanwhile, we propose a systematic approach,TacAlign, to narrow both structured and stochastic sources of domaingap, ensuring a reliable zero-shot sim-to-real transfer. We further evalu-ate Tac2Real on the contact-rich peg insertion task. The zero-shot trans-fer results achieve a high success rate in the real-world scenario, verifyingthe effectiveness and robustness of our framework. The project page is:https://ningyurichard.github.io/tac2real-project-page/