Towards Reliable Multi-Label Classification via Conditional Dependency Modeling

Multi-label classification models are commonly trained byassuming conditional independence among labels, typically through theuse of binary cross-entropy objectives. While this assumption simpli-fies optimization, it introduces a structural bias that ignores dependen-cies among labels and can adversely affect the calibration of predictedprobabilities. In this work, we theoretically analyze the impact of thisassumption and show that neglecting label dependencies leads to a mis-calibration effect that is linked to the conditional dependency of thelabels. Motivated by this observation, we propose Pairwise CorrelationDifference (PCD), a novel auxiliary loss designed to incorporate labeldependency information during training. PCD aligns the pairwise cor-relations of model logits with those of the ground-truth labels, therebyencouraging the network to capture conditional dependencies among la-bels. We combine PCD with the standard binary cross-entropy loss toform the Correlated Multi-Label Loss (CMLL), which serves as a objec-tive for dependency-aware training. We provide theoretical justificationfor the functional form of the proposed loss and its connection to thedependency structure of the label distribution. Extensive experimentson three benchmark multi-label datasets demonstrate that CMLL con-sistently produces better-calibrated predictions while maintaining com-petitive classification accuracy compared to existing multi-label learningobjectives. Additionally, we show that the proposed approach exhibitsrobustness to the choice of binning schemes used in calibration evalua-tion.