ATP-Bench: Towards the Agentic Tool Planning for MLLM Interleaved Generation

Interleaved text-and-image generation represents a signifi-cant frontier for Multimodal Large Language Models (MLLMs), offeringa more intuitive way to convey complex information. Current paradigmsrely on either image generation or retrieval augmentation, yet they typi-cally treat the two as mutually exclusive paths, failing to unify factualitywith creativity. We argue that the next milestone in this field is AgenticTool Planning, where the model serves as a central controller that au-tonomously determines when, where, and which tools to invoke to pro-duce interleaved responses for visual-critical queries. To systematicallyevaluate this paradigm, we introduce ATP-Bench, a novel benchmarkcomprising 7,702 QA pairs (including 1,592 VQA pairs) across eight cat-egories and 25 visual-critical intents, featuring human-verified queriesand ground truths. Furthermore, to evaluate agentic planning indepen-dent of end-to-end execution and changing tool backends, we proposea Multi-Agent MLLM-as-a-Judge (MAM) system. MAM evaluates tool-call precision, identifies missed opportunities for tool use, and assessesoverall response quality without requiring ground-truth references. Ourextensive experiments on 10 state-of-the-art MLLMs reveal that modelsstruggle with coherent interleaved planning and exhibit significant varia-tions in tool-use behavior, highlighting substantial room for improvementand providing actionable guidance for advancing interleaved generation.