QCA: Query- and Content-Aware Keyframe Selection for Long Video Understanding

Video understanding is often plagued by severe temporal re-dundancy, where processing dense frame sequences is both semanticallyinefficient and computationally expensive. This challenge is further am-plified when only a small subset of frames is truly relevant to the givenquery. In this paper, we propose a Query- and Content-Aware (QCA)keyframe selection framework that can select a compact yet information-rich set of frames from long videos. QCA first partitions the video intotemporal segments and estimates the information contribution of eachsegment by jointly modeling query relevance and content deviation, anddynamically allocates keyframe budget to each segment. Within eachsegment, QCA anchors on the most query-relevant frame and iterativelyincorporates additional frames to maximize diversity while maintaininghigh semantic relevance to the query. Crucially, our method requires noadditional training and can be seamlessly integrated into existing Video-LLMs. Extensive experiments across multiple long video understandingbenchmarks demonstrate that our proposed approach achieves state-of-the-art performance and has strong generalization ability. For instance,QCA achieves 67.8% on LongVideoBench using 128 frames, while GPT-4o achieves 66.7% using 256 frames. Our codes are available in GitHub.