Spectral Evolution-Guided Token Pruning in Large Multimodal Models

Reducing visual token redundancy is critical for acceleratingMultimodal Large Language Models (MLLMs) without degrading cross-modal reasoning performance. Existing token pruning methods typicallyrely on single-layer signals, such as attention scores or token similari-ties, which overlook the cross-layer transformation of visual representa-tions and may exhibit positional bias in multimodal token sequences.To address this limitation, we propose a training-free token pruningframework based on Cross-Layer Spectral Evolution (CLSE). Instead ofmeasuring token importance from single-layer feature magnitudes, CLSEquantifies how token representations evolve across Transformer layers inthe frequency domain. This evolution reflects the transition from high-frequency structural details to low-frequency semantic abstractions. Weobserve that tokens with stronger spectral redistribution across layers aremore likely to be semantically active and should therefore be preserved.By modeling cross-layer token dynamics, CLSE provides a stable im-portance criterion that mitigates positional bias. Extensive experimentson both image and video benchmarks demonstrate that CLSE achievesa superior trade-off between efficiency and accuracy under aggressivetoken reduction. Across multiple MLLMs, CLSE reduces FLOPs, KVcache memory, and latency while maintaining competitive or improvedperformance.