TinyHistory: Lightweight Video History Embeddings via Two-Stage Context Learning

History context is central to autoregressive video generation,driving consistency and storytelling for both commercial models andpersonal use cases. For example, personal users, offline workflows, andindividual-scale finetuning need to encode longer video histories undertight compute and memory budgets. We observe that content and identityconsistency is an essential requirement, and that complete, uninterruptedhistory coverage together with content query and interpretation capabil-ities is broadly desired. We present TinyHistory, a lightweight historyembedding learned through two-stage context learning. In the first stage,we pretrain the encoder on large-scale video data with a randomized framequery objective; in the second stage, we repurpose the pretrained encoderwithin an autoregressive video diffusion model to learn content-level con-sistency. As a result, we show that the learned lightweight embeddingsachieve consistency comparable (by VLM, VBench, ELO, etc.) to heavieralternatives, while reducing training overhead and extending the encod-able history length within a given memory budget. We conduct ablationstudies to analyze the influence and trade-offs of each component.