Language-Guided Transformer Tokenizer for Human Motion Generation

In this paper, we focus on motion discrete tokenization, whichconverts raw motion into compact discrete tokens—a process proven cru-cial for efficient motion generation. In this paradigm, increasing the num-ber of tokens is a common approach to improving motion reconstructionquality, but more tokens make it more difficult for generative models tolearn. To maintain high reconstruction quality while reducing genera-tion complexity, we introduce Language-Guided Tokenization (LG-Tok)for efficient motion tokenization. LG-Tok aligns natural language withmotion at the tokenization stage, yielding compact, high-level seman-tic representations. This approach not only strengthens both tokeniza-tion and detokenization but also simplifies the learning of generativemodels. Furthermore, existing tokenizers predominantly adopt convo-lutional architectures, whose local receptive fields struggle to supportglobal language guidance. To this end, we propose a Transformer-basedTokenizer that leverages attention mechanisms to enable effective align-ment. Additionally, we design a language-drop scheme, in which lan-guage conditions are randomly removed during training. This schemeprevents shortcut learning over text and enables the detokenizer to sup-port language-free guidance. On three generation benchmarks, LG-Tokoutperforms state-of-the-art methods (e.g., achieving an FID score of0.057 vs. MARDM’s 0.114 on HumanML3D). LG-Tok-mini uses onlyhalf the tokens while maintaining competitive performance, validatingthe efficiency of our semantic representations. Code and checkpoints areavailable at https://eanson023.github.io/LG-Tok/.