Hierarchical and Holistic Open-Vocabulary Functional 3D Scene Graphs for Indoor Spaces

Functional 3D scene graphs offer a versatile and flexible rep-resentation for 3D scene understanding and robotic manipulation, de-fined by object nodes, interactive elements, and functional relationshipedges. However, their potential remains underexplored due to the limitedcoverage of existing benchmarks and the overly straightforward designof previous pipelines, which primarily focus on large-scale furniture butlack of hierarchical structures. Therefore, in this work, we extend thebenchmark coverage by introducing dense tabletop objects and explicitmulti-level functional relationships. This expansion introduces criticalchallenges involving small-scale, dense, and similar instances, with lack ofvisual anchoring in relational reasoning, instance confusion during cross-frame fusion, and attribution uncertainty under dynamic viewpoints. Toaddress these issues, we propose an open-vocabulary pipeline based on 2Dvisual grounding and 3D graph optimization. Specifically, we anchor fine-grained functional edges from 2D visual evidence, and associate nodesacross frames in 3D using multiple cues. Furthermore, edge associationis formulated as temporal graph optimization, integrating evidence ac-cumulation, entropy regularization, and temporal smoothing to robustlydetermine the functional connections of each node. Finally, global hierar-chy shaping is performed to recover the hierarchical graph structure. Ex-tensive experiments demonstrate that the proposed method can reliablyinfer functional 3D scene graphs in challenging real-world scenes, therebyfurther unlocking their potential for practical applications. Code is avail-able at https://github.com/Hbelief1998/HHOpenFunGraph-ECCV26.