I've seen several ways to transform the data before feeding it into a neural network:
- Making it to fit the [0.; 1.] range by subtracting the
minand dividing by
(max - min);
- Making it zero-centered with unit variance by subtracting the mean and dividing by standard deviation.
keras.utils package also has its own normalization function
normalize(x, axis=-1, order=2), but the rationale, results and the purpose of this kind normalization is not clear to me. Moreover, this function can compute other kinds of norms, L1, L0 etc.
The majority of the pixels have the value of 0. The statistics for the data are:
DescribeResult(nobs=47040000, minmax=(0.0, 1.0), mean=0.13066062, variance=0.09493039, skewness=2.151106357574463, kurtosis=2.9144181812310546)
The statistics for L2-normalized data:
DescribeResult(nobs=47040000, minmax=(0.0, 0.27902707), mean=0.013792945, variance=0.0010852652, skewness=2.296393632888794, kurtosis=3.9849276836080234)
It looks like the data distribution has changed significantly.
I don't have enough knowledge in Linear Algebra to understand what has happened, and what L2-normalization actually does. So is it relevant to apply this kind of normalization to the input data, does it have any benefits and why should one prefer this way of normalization over traditional ones for training neural networks?