DiaDem: Advancing Dialogue Descriptions in Audiovisual Video Captioning for Multimodal Large Language Models

Accurate dialogue description in audiovisual video captioning is crucial for downstream understanding and generation tasks. However, existing models generally struggle to produce captions that faithfully reflect spoken content and speaker dynamics. To mitigate this limitation, we propose DiaDem, a powerful audiovisual video captioning model capable of generating captions with more precise dialogue descriptions while maintaining strong overall performance. We first design a dedicated pipeline to synthesize high-quality dialogue-aware audiovisual captions for supervised fine-tuning (SFT), and then introduce a difficulty-partitioned two-stage reinforcement learning (RL) strategy to further enhance dialogue descriptions. To enable systematic evaluation of dialogue description capabilities, we present DiaDemBench, a comprehensive benchmark designed to evaluate models across diverse dialogue scenarios, focusing on both speaker attribution accuracy and utterance transcription fidelity in audiovisual captions. Extensive experiments on DiaDemBench reveal even commercial models still exhibit substantial room for improvement in dialogue-aware captioning. Notably, DiaDem not only outperforms the Gemini series in dialogue description accuracy but also achieves competitive performance on general audiovisual captioning benchmarks, demonstrating its overall effectiveness. Our project is available at https://diadem-captioner.github.io/.