PA-VAD: Diffusion-Based Pseudo-Only Video Anomaly Detection via Domain-Aligned Memory Updates

Deploying video anomaly detection (VAD) in the real worldis often constrained by the scarcity, privacy, and cost of collecting realabnormal footage. We propose PA-VAD, a novel pseudo-only frame-work that trains an anomaly detector without using any real abnormalvideos, by pairing real normal videos with diffusion-synthesized pseudo-abnormal videos generated from a small set of real normal images. Be-yond proposing a generation-driven training pipeline, we make a keyempirical discovery: pseudo anomalies exhibit a characteristic spatiotem-poral magnitude bias in feature space, which can dominate MultipleInstance Learning and degrade generalization if left unaddressed. Tocounter this pseudo-induced bias, we introduce the Domain-Aligned Reg-ularized Module (DARM), which combines domain alignment with usage-aware memory updates to balance prototype coverage and stabilize opti-mization under biased pseudo supervision. Extensive experiments demon-strate that PA-VAD achieves 98.2% AUC on ShanghaiTech, 82.5% onUCF-Crime, and 95.1% on XD-Violence, and further improves general-ization to unseen anomaly classes in open-set evaluations. Notably, PA-VAD surpasses the best real-abnormal WVAD baselines on ShanghaiTechand XD-Violence by +0.6% and +0.9% , respectively, and improves overthe UVAD state of the art on UCF-Crime by +1.9% —showing that high-accuracy VAD is attainable without collecting real abnormal videos.