|
self.running_avg_mean = self.running_avg_mean + self.momentum * (batch_ch_mean.data.to(device) - self.running_avg_mean) |
As per the paper, https://arxiv.org/pdf/1702.03275.pdf, during inference in both functions, you should not update running averages. Otherwise you will diverge from the learned parameters over time.
Batch-Renormalization-PyTorch/batch_renormalization.py
Line 43 in 72bb6d9
As per the paper, https://arxiv.org/pdf/1702.03275.pdf, during inference in both functions, you should not update running averages. Otherwise you will diverge from the learned parameters over time.