EmbodiedHead: Real-Time Listening and Speaking Avatar for Conversational Agents

We present EmbodiedHead, a speech-driven talking headframework that equips LLMs with real-time visual avatars for conver-sation. A practical embodied avatar must achieve real-time generation,unified listening-speaking behavior, and high rendered visual quality si-multaneously. Our framework couples the first Rectified-Flow DiffusionTransformer (DiT) for this task with a differentiable renderer, enablingdiverse, high-fidelity generation in as few as four sampling steps. Priorlistening-speaking methods rely on dual-stream audio, introducing an in-terlocutor look-ahead dependency incompatible with causal user–LLMinteraction. We instead adopt a single-stream interface with explicitper-frame listening-speaking state conditioning and a Streaming AudioScheduler, suppressing spurious mouth motion during listening while en-abling seamless turn-taking. A two-stage training scheme of coefficient-space pretraining and joint image-domain refinement further closes thegap between motion-level supervision and rendered quality. Extensive ex-periments demonstrate state-of-the-art visual quality and motion fidelityin both speaking and listening scenarios.