todayToday I had a discussion about the right way to normalize data, especially image data. The standard approach, as found in many tutorials etc., seems to be following:
For. For example, the data has a range of integer values from k = 0, ..., 255$k = 0, ..., 255$ (e.g. uint8 value for each channel or grayscale in image encoding), then divide by with (N-1)=255, where N=256 is the total number of possible discrete$N=256$ values. The resultingWe form the scaled values:
$$x = \frac{k}{N-1} \quad \quad \rightarrow \quad \quad x = 0, \tfrac{1}{N-1}, ..., \tfrac{N-2}{N-1}, 1.$$
These value x = k/(N-1) is now inare on a rangescale between 0.0 and 1.0, with 0.0 for k=0,zero and 1.0 for k=255one. After After processing the x$x$ values by a neural network for example, the xprocessed values $\hat{x}$ get transformed back to k = round(x*(N-1)), where round() is$\hat{k} = \hat{x} \cdot (N-1)$ and then rounded to the nearest integer. This gives us processed values for the nearest integer rounding$k$ values.
My argument then was: ThisProblem with this procedure: I believe that this procedure leads to a systematic statistical distortion. For For example: lets say N=4, then the original integer get placed at x = 0, 1/3when $N=4$, 2/3 and 1the scaled values are $x = 0, \tfrac{1}{3}, \tfrac{2}{3}, 1$ for k = 0, 1, 2, 3$k = 0, 1, 2, 3$. The x then get We process these scaled values to some processed internally,values $\hat{x}$ and now lets assume, that before transformingthen we convert these back to integerprocessed values of $k$ via rounding, the x are uniformly distributed.so that:
$$\hat{k} = \hat{k}(\hat{x}) = \begin{cases} 0 & & \text{for }\hat{x} \in [0, \tfrac{1}{6}), \\ 1 & & \text{for }\hat{x} \in [\tfrac{1}{6}, \tfrac{3}{6}), \\ 2 & & \text{for }\hat{x} \in [\tfrac{3}{6}, \tfrac{5}{6}), \\ 3 & & \text{for }\hat{x} \in [\tfrac{5}{6}, 1]. \\ \end{cases}$$
Now the interval [0, 1/6] will be mapped to k=0let's assume, that the interval [1/6, 3/6] to k=1, [3/6, 5/6] to k=2, and [5/6, 1] to k=3processed values $\hat{x}$ are uniformly distributed. Hence, Then the probabilities for k are: p(0)=1/6, p(1)=1/3, p(2)=1/3 and p(3)=1/6, whichof the corresponding values of $\hat{k}$ do not reflect the originally uniformly distributed x values.
My proposed solution: Because of thatthis problem, my proposal wasis to normalize by an alternative approachscale the values as:
Normalization: x = k/N + 1/(2*N)$$x = \frac{k + \tfrac{1}{2}}{N} \quad \quad \rightarrow \quad \quad x = \tfrac{1}{2N}, \tfrac{3}{2N}, ..., \tfrac{2N-3}{2N}, \tfrac{2N-1}{2N}.$$
By this proceduremethod, at first the original integerscaled values get placed in the midsmidpoints of N equal intervalls.
Denormalization: k = round(N*x-1/2)
By this denormalization all$N$ equal N intervals get mapped, and we take $\hat{k} = \hat{x} \cdot N - \tfrac{1}{2}$ and then round to the correspondingnearest integer to get back the processed values k=0,...,N-1.of (only for x=1.0: N*x-1/2=3$k$.5, round (3.5)=4Since - this case has to be processed separately, since [-0.5, +0.4999...]$1 \cdot N - \tfrac{1}{2}$ gets rounded up to 0.0$N$, which is rather a technical aspect. In all other cases,out of the back transformation works properlyrange of allowable values, we process this case separately.)
The The transformation back to integer values of $\hat{k}$ then leads to equal probabilities for all values, in the example above: p(0)=p(1)=p(2)=p(3)=1/4.
My question: In my opinion, this seems to be much more consistent in a statistal senseesense. So So my question is:, what do you think about it, and why is the first approach (dividing by N-1$N-1$) so common?
