StreamTalk: Streaming Co-Speech Gesture Generation with Key-Pose Anchoring

Real-time co-speech gesture generation requires producing 3D motion clip by clip as speech streams in. Existing methods are fundamentally open-loop: each clip is synthesized conditioned only on past context, with no mechanism to verify or correct trajectory plausibility. Small per-clip errors therefore compound silently, causing the well-known drift problem--motion that gradually deviates from natural distributions over minute-scale horizons. We make a key observation: drift stems not from poor local motion quality--modern diusion models already produce convincing short clipsbut from the absence of forward constraints that tell each clip where it should arrive. Supplying even a single plausible key pose at a clip's tail as a destination anchor is sufficient to suppress drift dramatically. Building on this insight, we propose StreamTalk, a closed-loop streaming framework that introduces a periodic generate-retrieve-refine feedback cycle. At inference, Streaming Pose-Guided Generation (SPG) first produces a coarse clip, retrieves a plausible tail key pose from a speaker-specic motion database, and refines the clip with this anchor before passing it to the next window. To enable the model to exploit such sparse anchors effectively, we introduce Stochastic Anchor Masking (SAM) during training, which independently masks random pose and translation frames so the model learns to inpaint complete motion from partial boundary conditions. A part-aware DiT architecture further disentangles hand, body, and translation streams to prevent global displacement from interfering with local articulation. Extensive experiments on the BEAT2 benchmark demonstrate that StreamTalk achieves state-of-the-art motion quality (FGD), signicantly suppresses long-horizon drift compared to open-loop baselines, and runs in real time at 76 FPSenabling practical minute-scale streaming co-speech gesture generation. Code will be released upon acceptance.