From Predictions to Embeddings: Dual Knowledge Distillation for Instance-Dependent Partial Label Learning

Partial label learning in the instance-dependent setting (ID-PLL) poses significant challenges, as candidate labels are highly corre-lated with sample features, and strong semantic overlap among candi-dates often leads to error-prone disambiguation. Meanwhile, the infor-mative dark knowledge inherent in non-candidate labels remains largelyunderexplored. To address these issues, we propose DKDP, a dual dis-tillation method that extracts and transfers more stable and reliableknowledge from the teacher model. Specifically, we first use constrainedprojection to structurally rectify the teacher’s prediction distribution,enforcing the prior that the minimum value of candidate labels exceedsthe maximum value of non-candidate labels. On this basis, we guide thestudent model to fit the corrected teacher outputs. We further derive theminimum modification required to satisfy the prior, which theoreticallyguarantees the interpretability of the rectification process. Moreover, weintroduce class prototypes to model the distribution of fine-grained se-mantic relations, extending reliable knowledge distillation from the out-put space to the embedding space. This yields a stable, separable geomet-ric structure in the feature space. Extensive experiments on benchmarkdatasets demonstrate the effectiveness of our proposed method.