Mimicking Radiologists: A Coarse-to-Fine Framework with Structural Sparse Tokens for Dual-LLM Computed Tomography Report Generation

Computed tomography report generation (CTRG) automatesradiology reporting to reduce clinical workload and facilitate patient care.Recent efforts in applying the rapidly developing large language mod-els (LLMs) have advanced the field; yet, they still face a fundamentalchallenge due to the large volume of 3D data: effectively reducing highfeature redundancy and computational burden while simultaneously ex-tracting information-rich representations. To address this challenge, thiswork presents a novel CTRG framework that fully imitates the coarse-to-fine visual search pipeline practiced by radiologists. Our frameworkfirst employs a ViT-based image-text alignment architecture to extracta global token and local patch tokens for each anatomical structure, en-hanced by a mask-guided sparse negative-entropy loss. An abnormality-proposal LLM then processes the global token and clinical metadata topropose a shortlist of candidate abnormalities. This shortlist promptsan abnormality-prompted local token filter (AP-LTF) to select the mostinformative patch tokens, effectively reducing redundancy while preserv-ing critical information. Finally, a report-generation LLM takes in theglobal token, selected local tokens, and clinical metadata to composea full report. During training, ground-truth abnormalities are prefixedto the reference report to enhance awareness of clinic-relevant findings,and group relative policy optimization (GRPO) aligns the abnormality-proposal and report-generation LLMs for collaborative efficacy. Exper-imental results on two public CTRG datasets demonstrate the supe-rior performance of our framework compared to existing state-of-the-artFig. 1: Comparison of token compression strategies for CTRG. (a) 3D spatial pooling[2, 8] achieves high compression rate but incurs significant information loss. (b) Mask-guided volume cropping [12, 22] yields region-specific tokens but omits intra-regionredundancy. (c) [26] selects the top K tokens with the largest attention weights toregion-specific queries, yet may be subject to dominant “sink tokens” [14, 29]. (d) Ourmethod first proposes a shortlist of candidate abnormalities using only a single globaltoken per structure and clinical metadata. Then, an abnormality-prompted local tokenfilter (AP-LTF) combines the shortlist, a learned score, and the attention weight for acomprehensive token selection, thereby preventing sink tokens from dominating.methods in terms of clinical efficacy, RaTEScore, and GREEN scores.Ablation studies further validate the effectiveness of its novel design.