C3-Bench: A Context-Aware Change Captioning Benchmark

While Change Captioning systems have garnered substantialattention to respond to our evolving world, their true performance on di-verse real-world change contexts remains largely unexplored due to thelack of comprehensive evaluation frameworks. To fill this gap, we proposeC3-Bench, a comprehensive benchmark for evaluating Context-awareChange Captioning. C3-Bench features: (1) 4,996 human-labeled imagepairs of 51 real-world change contexts across four domains (e.g., naturalscenes, remote sensing imagery, image editing, and anomalies), each withdiverse, carefully curated scenarios derived from multiple change-centriccommunities; and (2) the first LLM-as-Judge evaluation framework in thechange captioning task that measure fine-grained dimensions (e.g., cor-rectness, specificity, fluency, and relevance), along with a novel reversibil-ity metric exploring whether models understand changes with symmetricconsistency. Based on C3-Bench, we benchmark 32 models—including* Corresponding authorconventional change captioning models, proprietary Large MultimodalModels (LMMs), and 2B-90B open-source LMMs. We reveal a fundamen-tal blind spot in the prevailing change captioning paradigm: Once thechange context departs from training-style regimes, conventional mod-els collapse, and even state-of-the-art LMMs such as GPT-5.2 exhibitsystematic domain- and position-dependent errors that distort reliablechange understanding. By making these hidden failure modes explicitand measurable, we delineate the next frontier for building generalizableand trustworthy change captioning systems. All codes and datasets arepublicly available on the project page.