Exploring Efficient Reasoning Segmentation with Small Language Models

Reasoning segmentation is a challenging vision–language taskthat performs pixel-level segmentation guided by language reasoning overimplicit textual descriptions. Existing methods typically employ largelanguage models (LLMs) to achieve such reasoning capability, yet theirsubstantial computational and memory demands limit practical deploy-ment. To overcome this, we present LReSeg, the first attempt to tacklereasoning segmentation with a small language model (SLM), achievingstrong performance with substantially reduced model size and computa-tional cost. To address the challenges of scaling reasoning segmentationto SLMs, we propose two key designs: First, unlike LLMs, SLMs havelimited capacity to provide sufficient spatial instruction cues for mask de-coding. We introduce register tokens that aggregate text-conditioned spa-tial features via the SLM’s self-attention and inject them into the visualtoken stream to enrich the mask decoder’s inputs. Second, we adopt aunified encoder architecture to eliminate the redundant visual backboneof conventional dual-encoder designs, naturally ensuring feature consis-tency between reasoning and mask decoding. LReSeg surpasses modelsof similar scale with superior efficiency, while being nearly 10× smaller(800M vs. ∼8B) and 3.3× faster than conventional 7B-scale LLM-basedmethods. Code is available at https://github.com/downdric/LReSeg.