Bayesian Self-Attention with Local Pixel Correlations for Lightweight Denoising Transformers

Transformer architectures have achieved strong performancein image denoising, but their computational and memory costs remainhigh. We revisit self-attention (SA) from a Bayesian perspective and showthat standard SA mainly exploits first-order inter-patch statistics, resem-bling a learnable non-local averaging scheme. However, this formulationneglects second-order intra-patch statistics, which are important for cap-turing local pixel dependencies. To address this, we introduce a BayesianSA formulation that jointly models first-order inter-patch and second-order intra-patch statistics. This leads to a lightweight denoising Trans-former, termed NLformer, featuring a dual-branch attention design andan efficient feed-forward module. Experiments on several benchmarksshow that NLformer outperforms existing lightweight denoising networksand substantially narrows the gap to full-size Transformer-based denois-ers while maintaining low model complexity.