The Label Imitation Game: Turing Test Network for Zero-Shot Pseudo-Label Pruning

Foundation model pseudo-labeling—labeling data strictly viazero-shot inference—enables massive scale, but performance is under-mined by hallucinations that evade standard thresholds. To eliminatethese errors, we introduce the Turing-inspired Label Imitation Game(LIG), a framework that formalizes pseudo-label pruning as an adversar-ial interrogation. Rather than filtering labels via isolated thresholds, weuse the LIG to train a Turing Test Network (TTN), a task-agnostic“judge” that evaluates candidate pseudo-labels within a dataset-wide con-text. Experiments across four diverse datasets demonstrate the TTN’srobustness, consistently enhancing label accuracy for three state-of-the-art vision-language models without costly supervision or retraining. Cru-cially, we demonstrate that learned semantic-contextual logic is a robustalternative to spatial-geometric verification, enabling a unique zero-shottask transfer capability—a TTN trained strictly on image classificationdatasets can effectively prune complex object detection pseudo-labels.This pruning yields F1 -score gains of 28% for the worst-performing base-line categories and 44% with task-specific fine-tuning. Significantly, wealso observe Category Revival, where the TTN pruning “detoxifies” thetraining signal for downstream models and enables them to recover fromzero recall on transfer-vulnerable classes. The pre-trained TTN modelsand code are available at https://github.com/voxel51/ttn.