Director: Instance-aware Gaussian Splatting for Dynamic Scene Modeling and Understanding

Volumetric video seeks to model dynamic scenes as tem-porally coherent 4D representations. While recent Gaussian-based ap-proaches achieve impressive rendering fidelity, they primarily emphasizeappearance but are largely agnostic to instance-level structure, limit-ing stable tracking and semantic reasoning in highly dynamic scenarios.In this paper, we present Director, a unified spatio-temporal Gaussianrepresentation that jointly models human performance, high-fidelity ren-dering, and instance-level semantics. Our key insight is that embeddinginstance-consistent semantics naturally complements 4D modeling, en-abling more accurate scene decomposition while supporting robust dy-namic scene understanding. To this end, we leverage temporally alignedinstance masks and sentence embeddings derived from Multimodal LargeLanguage Models to supervise the learnable semantic features of eachGaussian via two MLP decoders, enabling language-aligned 4D repre-sentations and enforcing identity consistency over time.To enhance temporal stability, we bridge 2D optical flow with 4D Gaus-sians and finetune their motions, yielding reliable initialization and re-ducing drift. For the training, we further introduce a geometry-awareSDF constraints, along with regularization terms that enforces surfacecontinuity, enhancing temporal coherence in dynamic foreground model-ing. Experiments demonstrate that Director achieves temporally coher-ent 4D reconstructions while simultaneously enabling instance segmen-tation and open-vocabulary querying.