Seeing Fast and Slow: Learning the Flow of Time in Videos

How can we tell whether a video has been sped up or sloweddown? How can we generate videos at different speeds? Although videoshave been central to modern computer vision research, little attentionhas been paid to perceiving and controlling the passage of time. In thispaper, we study time as a learnable visual concept and develop modelsfor reasoning about and manipulating the flow of time in videos.We first exploit the multimodal cues and temporal structure naturallypresent in videos to learn, in a self-supervised manner, to detect speedchanges and estimate playback speed. We then show that these learnedtemporal reasoning models enable us to curate the largest slow-motionvideo dataset to date from noisy in-the-wild sources. Such slow-motionfootage, typically filmed by high-speed cameras, contains substantiallyricher temporal detail than standard videos. Using this data, we furtherdevelop models capable of temporal control, including speed-conditionedvideo generation, which produces motion at specified playback speed,and temporal super-resolution, which transforms low-FPS, blurry videosinto high-FPS sequences with fine-grained temporal details. Our findingshighlight time as a manipulable, perceptual dimension in video learning,opening doors to temporally controllable video generation, temporalforensics detection, and potentially richer world-models that understandhow events unfold over time. Project page: https://seeing-fast-and-slow.github.io/.