Humanoid Whole-Body Manipulation via Active Spatial Brain and Generalizable Action Cerebellum

In this paper, we explore spatial-aware humanoid whole-body manipulation task. Compared with tabletop settings, this taskposes two key challenges: 1) Spatial understanding is challenging in com-plex 3D environments with diverse spatial relations. 2) Action generationis difficult to generalize, as limited and costly real-robot data restrictsdata-driven models generalization. To address these challenges, we pro-pose a generalizable humanoid loco-manipulation framework that lever-ages the spatial perception and action generation capabilities of multi-agent large models. Specifically, our framework includes two components:Active Spatial Brain for active spatial perception and decision-making,and Generalizable Action Cerebellum for executable robot action genera-tion. The first component actively perceives the spatial scene and makesdecisions on task planning and subtask decomposition. The second com-ponent generates executable robot actions based on the decisions madeby the first module without needs of task-specific real robot data. Tobenchmark our framework, we design a set of spatial manipulation tasksfrom two perspectives: evaluating spatial perception and understand-ing, and assessing real-robot task performance. The results demonstratestrong performance on both aspects across diverse tasks and environ-ments.