Seeing Isn't Orienting: A Cognitively Grounded Hierarchical Benchmark for Object Orientation in MLLMs

Humans develop object orientation understanding progressively, from recognizing which way an object faces, to mentally rotating it, to reasoning about how multiple objects are oriented relative to each other. Yet current vision-language benchmarks treat orientation as an afterthought, conflating it with positional relationships and general scene understanding. We introduce Discriminative Orientation Reasoning Intelligence (DORI), a cognition-informed hierarchical benchmark that establishes object orientation as the primary evaluation target. Specifically, DORI decomposes orientation into four dimensions, each assessed at both coarse (categorical) and granular (metric) levels. This results in 33,656 multiple-choice questions over diverse open-vocabulary objects within 13,652 real-world and synthetic images taken from 14 sources. DORI’s coarse-to-granular design isolates orientation from confounds such as object recognition difficulty, scene clutter, and linguistic ambiguity through bounding-box isolation, standardized spatial reference frames, and structured prompts. Our evaluation of 26 state-of-the-art vision-language models reveals a consistent pattern: models competent on general spatial benchmarks remain near-random on object-centric orientation tasks. Even the best models achieve only 64.2% on coarse and 42.9% on granular judgments, with the largest drops on compound rotations and inter-object reference frame shifts. Large coarse-to-granular gaps further expose that models rely on categorical heuristics rather than geometric reasoning, a limitation invisible to existing benchmarks. These findings establish orientation understanding as an unsolved challenge in multimodal systems, with direct implications for robotic manipulation, 3D scene reconstruction, and human-AI interaction. Find the dataset here: https://huggingface.co/datasets/appledora/DORI-Benchmark