Towards Temporal Compositional Reasoning in Long-Form Sports Videos

Sports video analysis is a challenging domain for multimodalunderstanding because it involves complex and dynamic human activi-ties. Despite rapid progress in Multimodal Large Language Models (MLLMs),long-horizon reasoning in sports videos remains difficult, as answeringquestions requires both locating and integrating temporally sparse evi-dence into reasoning. We attribute this limitation to two closely relatedfactors: insufficient supervision over temporally dispersed evidence andthe lack of methods for explicit temporal evidence localization and justi-fication. To address these gaps, we introduce SportsTime, a large-scalebenchmark for long-form sports video understanding, comprising 14K+open-ended QA pairs and 50K+ step-wise temporal evidence annota-tions. Building on SportsTime, we propose Chain-of-Time Reasoning(CoTR), which treats reasoning as a process of temporally groundedevidence composition. Specifically, during training, CoTR introduces atemporal-reward GRPO to encourage temporally grounded reasoning.During inference, it employs an anchor-observe-infer evidence-seekingloop to iteratively localize, verify, and compose temporal evidence beforeproducing the final answer. Experiments show that SportsTime exposessubstantial gaps in current MLLMs, while CoTR yields consistent gainsover strong baselines, improving both temporal compositional reasoningperformance and step-wise grounding quality. The dataset and code areavailable at https://github.com/ustiniansy/SportsTime.