Brain-inspired Lab Peking University

CaRe-BN: Precise Moving Statistics for Stabilizing Spiking Neural Networks in Reinforcement Learning

spiking neural networks reinforcement learning batch normalization

Zijie Xu, Xinyu Shi, Yiting Dong, Zihan Huang, Zhaofei Yu. CaRe-BN: Precise Moving Statistics for Stabilizing Spiking Neural Networks in Reinforcement Learning. The Fourteenth International Conference on Learning Representations (ICLR), 2026.*

Read the paper

Research Background and Problem

Background

Spiking Neural Networks (SNNs) provide highly energy-efficient and low-latency decision-making capabilities by mimicking the event-driven dynamics of biological neurons, making them ideal for neuromorphic hardware.

Limitation of Existing Methods

However, traditional Batch Normalization (BN) suffers a severe breakdown in online Reinforcement Learning (RL) because moving statistics cannot be accurately estimated under continually shifting data distributions, leading to unstable training and suboptimal policies.

Our Perspective

To overcome these challenges, we propose Confidence-adaptive and Re-calibration Batch Normalization (CaRe-BN), a novel normalization strategy tailored specifically to provide precise statistics and stabilize SNN-based RL without disrupting the training process.

figure image

Contributions

  • Introduces Ca-BN (Confidence-adaptive update), a confidence-weighted moving estimator that dynamically adjusts statistics estimation based on the reliability of the current approximation to ensure unbiasedness and reduce variance.
  • Introduces Re-BN (Re-calibration), a periodic correction scheme that leverages replay buffer resampling to refine inference statistics and correct accumulated estimation errors.
  • Maintains the inherent energy efficiency of SNNs; the CaRe-BN layer is seamlessly fused into synaptic weights after training, introducing zero additional inference overhead during deployment.
  • Achieves state-of-the-art results, improving SNN performance by up to 22.6% across different spiking neuron models and RL algorithms, and remarkably outperforming Artificial Neural Network (ANN) counterparts by 5.9%.

Core Method

The CaRe-BN framework integrates two complementary mechanisms to enable precise, low-variance estimation of BN statistics under the nonstationary dynamics of RL. First, the Ca-BN module is applied at every update step, acting as a confidence-guided mechanism that adaptively reweights estimators to minimize the mean-squared error of BN statistics when distribution shifts occur. Second, because online estimates may still drift due to stochastic mini-batch noise, the Re-BN module is executed periodically. It draws aggregated calibration batches from the replay buffer to compute exact statistics, effectively correcting accumulated bias without requiring millions of forward passes.

figure image

Representative Results

Extensive evaluations across discrete action spaces (Atari benchmarks) and continuous control tasks (MuJoCo suite) demonstrate that CaRe-BN significantly resolves the issue of imprecise moving statistics. SNN-based agents equipped with CaRe-BN consistently achieved higher exploration returns, converged faster, and exhibited significantly lower variance compared to standard SNNs. Notably, when paired with the TD3 algorithm, CaRe-BN allowed simple SNNs to surpass the performance of traditional ANN-based RL agents by an average of 5.9%, establishing a new state-of-the-art normalization strategy that bridges the gap between neuromorphic efficiency and high-performance RL.

figure image