TecoPrompt: Temporal-Conservative Prompt Learning for Vision-Language Models

Prompt learning adapts vision-language models, such as CLIP,by adjusting a small set of context tokens. However, under few-shot su-pervision, even moderate label noise can disrupt prompt optimization. Toaddress this issue, we propose TecoPrompt, a closed-loop robust prompt-learning framework that revisits optimal transport (OT) pseudo-labelingfrom a temporal perspective. TecoPrompt employs an entropic OT planin the CLIP semantic space to obtain globally consistent label candidates.It verifies the reliability of these candidates by examining trajectorystability: a noisy label is only rewritten if the OT candidate remainsunchanged within a K-epoch temporal stability window and passes a con-fidence gate based on Exponential Moving Average (EMA). This approachhelps reduce confirmation bias. The rewritten labels are then integratedback into prompt training using a tri-group objective that includes threeloss functions aligned with clean, mid, and noisy subsets. Experimentson seven datasets with synthetic symmetric and asymmetric noise, aswell as Food101N, demonstrate significant performance improvements.For example, on the OxfordPets dataset, with 50% asymmetric noise,TecoPrompt achieves an accuracy of 0.843, up from 0.775.