Atlas is Your Perfect Context: One-Shot Customization for Generalizable Foundational Medical Image Segmentation

Accurate anatomical structure segmentation in medical im-age is essential for diagnosis and treatment planning. While recent inter-active segmentation foundation models enhance generalization throughlarge-scale multimodal pretraining, they still depend on precise promptsand can fail in underrepresented clinical contexts (e.g., small organs-at-risk). We present AtlasSegFM, an atlas-guided framework that cus-tomizes off-the-shelf foundation models to new clinical contexts with asingle annotated example. AtlasSegFM 1) performs atlas-query regis-tration to generate context-aware prompts, 2) refines the segmentationwith a frozen foundation model, and 3) applies a lightweight adaptivefusion module to combine atlas priors with foundation-model inputs andpredictions. Extensive experiments on six public and in-house datasetsacross radiotherapy and vascular scenarios show consistent gains, withthe largest improvements on small and delicate structures. AtlasSegFMprovides a lightweight, deployable solution for one-shot customization ofsegmentation foundation models in real-world clinical workflows.