DETRAM: End-to-end DEtection, Tracking and Recovery of HumAn Meshes

In the task of human mesh recovery (HMR), multi-personscenes are particularly difficult to handle due to the many entities thatappear and occlusions between them over time. In particular for videoinputs, there is a need to track each entity reliably and consistently.Existing methods rely on pretrained human detection modules, increas-ing their runtime and limiting the number of tracked entities. We presentDETRAM, a unified framework for multi-person HMR and tracking thatsimultaneously detects, reconstructs, and tracks humans across time,both automatically and via user prompts. DETRAM uses a single trans-former decoder with an identity-consistent set of learnable query embed-dings that persist across frames: detection queries discover new people,tracking queries maintain pose and shape for existing individuals, andprompt queries follow user-specified identities. Our approach achievesstate-of-the-art tracking results on PoseTrack21, 3DPW, BEDLAM, andMuPoTS-3D, and competitive reconstruction accuracy on BEDLAM and3DPW, while uniquely supporting prompt-based tracking of individualsin multi-person scenes. To our knowledge, this is the first method tounify promptability and multi-person HMR with tracking in an end-to-end trainable framework, enabling user-directed human analysis in