LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning?

Many real-world applications of spatial intelligence, such asrobotic control, autonomous driving, and automated assembly, requirespatial reasoning across multiple sequential steps. However, the extent towhich current Multimodal Large Language Models (MLLMs) possess thiscapability remains largely unexplored. Inspired by LEGO construction, arecreational activity that critically relies on multi-step spatial reasoning,we introduce LEGO-Puzzles: a benchmark designed to systematicallyevaluate the spatial reasoning capabilities of MLLMs from basic spatialunderstanding to multi-step planning. LEGO-Puzzles contains two tasksets. The Elementary set covers 11 visual question-answering (VQA)tasks with 1, 100 carefully curated samples to test elementary spatialreasoning skills that are cruical for LEGO assembly. The Planning setdirectly requires the model to generate a step-by-step plan for assemblinga target LEGO structure, where the tasks are organized into subsets withdifferent planning horizons ranging up to 8. Our evaluation of 29 state-of-the-art MLLMs shows that even the strongest models struggle withelementary reasoning tasks in LEGO construction, falling at least 20%behind human performance. The planning accuracy also quickly dropsto 0% as the number of planning steps increases, whereas our humanparticipants solve all the tasks perfectly. Switching the output formatfrom multiple choice to image generation degrades model performanceeven further, leading to zero accuracy even for planning 3 steps. Over-all, LEGO-Puzzles reveals critical limitations in current MLLMs’ spatialreasoning capabilities and highlights the need for substantial advances.