In section 3 of the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, by Sergey Ioffe and Christian Szegedy, it says
Since the full whitening of each layer's inputs is costly and not everywhere differentiable, we make two necessary simplifications.
I understand that the full whitening is related to their analysis in section 2, however, I still couldn't figure out the difference between the full whitening and batch normalization. I think they are basically the same -- subtracting the mean from the inputs and then dividing by the variance. Where am I wrong?