I was reading the new layer normalization (LN) paper and it mentioned that batch normalization (BN) batch normalization required moving averages. I was re-reading the paper but and it says the moving averages are needed to estimate the accuracy of the model as it trains (by accuracy I take it to mean train, validation, test errors/accuracy). I take the reason for this is so to save time since it probably estimates accuracies of mini batches and just averages it as it goes until the end.
However, the LA paper seems to imply (in the introduction) that activation accuracies are used in BN for some other reason. Why would BN require to store activation statistics at the end of training?