PolyLayout: Multi-room Manhattan Layout Estimation

Estimating room layouts from multi-view imagery is a coretask for indoor scene understanding. Existing methods are typically lim-ited either by poor generalization to new datasets or restrictive geometricassumptions of the room shape or camera configuration. Most also es-timate rooms independently, failing to exploit shared building structuresuch as dominant directions, ground plane or ceiling height.We propose PolyLayout, a multi-room layout estimation method that pa-rameterizes room layouts as Manhattan 3D polygons and optimizes themjointly across multiple rooms. The optimization objective is predicted bya neural network on top of robust pre-trained visual features and trainedend-to-end with supervision only on output room layouts. At the sametime, camera projection and polygon updates remain explicit and model-based. This separation between learned scoring and geometry improvesgeneralization to new datasets and camera parameters. During optimiza-tion, PolyLayout adaptively refines the polygon topology through iter-ative wall split and merge operations while jointly utilizing structuralcues across rooms. We introduce two new multi-view multi-room layoutbenchmarks by providing layout annotations to existing datasets, andexperiments show that PolyLayout outperforms prior approaches, bothin terms of accuracy and robustness.Project page: https://ghanning.github.io/PolyLayout