CARE: Causally-Aligned Reasoning Exploration for Medical Large Language Models

Large Language Models (LLMs) have shown strong potentialfor medical reasoning, yet the scarcity and cost of expert-annotated dataconstrain their progress. While reinforcement learning offers a scalable al-ternative, standard outcome-based methods in medicine often suffer fromautoregressive credit assignment failure and gradient variance explo-sion. This leads to the “Right Answer, Wrong Reason” trap, where mod-els inadvertently reinforce spurious correlations and dataset shortcutsrather than valid clinical deduction. In this work, we propose Causally-Aligned Reasoning Exploration (CARE), a theoretically groundedframework for intrinsic experience curation. CARE is built upon tworigorous conditions for high-quality training trajectories: Causal Suf-ficiency, which utilizes an agreement-based self-verification mechanismto mimic do-calculus interventions and effectively debias gradients; andProximal Learnability, which employs dynamic entropy bounds to se-lect experiences within the model’s zone of proximal development forvariance-bounded optimization. These rigorously filtered experiences areoptimized via a dual-stream objective that combines on-policy group-relative exploration with difficulty-weighted experience replay. Extensiveexperiments on diverse medical multimodal and text-only benchmarksdemonstrate that CARE consistently outperforms other strong competi-tors, substantially reducing correct-but-inconsistent reasoning and im-proving training stability.