# Why does normalizing image twice work?

I made a 'mistake' while training a neural network, it is a typical image classification problem like this. However the data is much larger and came from Kaggle.

In my Dataset class from PyTorch, I defined a flag

if self.transform_norm is False:
image = image.astype(np.float32) / 255.0


and this would signify that if my augmentations pipeline does not have a normalization technique, then we set this flag to False. One example that I would set the above flag to be True the below augmentation appear in my pipeline:

albumentations.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0)


I forgot to set the flag to True and thus, the images first went through a standardization from [0,255] to [0,1] and then normalized using mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]. I thought I did wrong but the training results were actually good. So I dug deeper and found PyTorch's documentation and realized that it may be me who has been doing it all wrong? However, I am not using PyTorch's pretrained model out of the box, usually, I go to Ross's timm/geffnet for the models. Do let me know if there is a "right" approach.

All pre-trained models expect input images normalized in the same way, i.e. mini-batches of 3-channel RGB images of shape (3 x H x W), where H and W are expected to be at least 224. The images have to be loaded in to a range of [0, 1] and then normalized using mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225]. You can use the following transform to normalize:

• The quote seems to describe what you’ve done. What part of it makes you uncertain? – Sycorax Dec 28 '20 at 14:20
• Thanks! The uncertainty comes from the part whereby for my past trainings (many of them) I have almost never intentionally do both /255 and normalization as described above at the same time. I always thought it is either or. But I realised I may be wrong. – nan Dec 28 '20 at 14:50
• I agree that this is unusual, but it seems that, in this particular case, the documentation is very explicit about scaling in this exact way. As a matter of mathematics, linear functions are closed under composition, so if you really wanted to, you could roll both steps into a single step using a little arithmetic. – Sycorax Dec 28 '20 at 15:45
• I agree, thanks for your reply. I was actually shocked that the end results are good. (I am not using the pretrained models from the documentation so I don’t think I need to obey this). But I’m just confused on why it worked. – nan Dec 28 '20 at 15:52
• So perhaps you could edit your question to ask "Why does this work?" instead of asking whether it's right, since you have documentation that very plainly states that this is correct. – Sycorax Dec 28 '20 at 16:06

It's not normalized twice! It's normalized once, using two steps.

I say it's normalized once because all the normalization does is apply two linear transformations. We can rewrite two or more linear transformations as a single linear transformation. All we need to do is to choose a certain linear function to transform the raw pixel values to the values that the neural network expects to receive. We can show that the two-step process achieved in code is exactly the same as an equivalent operation carried out in one step.

The network expects to receive images with a certain scale, but images are encoded by with values between 0 and 255. The route suggested in the documentation is two-step.

1. For some pixel $$p\in[0,255]$$, divide by 255: $$q = \frac{p}{255}$$. If we wanted to emphasize that this is a "linear scale and shift" transformation, we could even write $$q = \frac{p-0}{255}$$.
2. Subtract the mean and divide by the standard deviation: $$z = \frac{q - \mu}{\sigma}$$. The network expects to receive $$z$$ as inputs.

We can do this in one step instead, because the composition of linear functions is linear. Just doing substitution and rearranging, we can show

\begin{align} z&=\frac{q - \mu}{\sigma} \\ &=\frac{p/255 - \mu}{\sigma} \\ &=\frac{p - 255\mu}{255\sigma} \\ \end{align}

So in the specific case, you can achieve the exact same scaling using the transformation albumentations.Normalize(mean=[255*0.485, 255*0.456, 255*0.406], std=[255*0.229, 255*0.224, 255*0.225]). This should make intuitive sense, because we're just rescaling the transformation to take place on the $$[0,255]$$ interval of pixel values, instead of a $$[0,1]$$ interval.

• I understood perfectly now thanks! But just one more question, why did my accidental normalization "thrice" worked, I went through one division of 255, followed another Normalize() which is another division of 255, and then applying the mean and std. So technically, if my image pixel is 255, I went through 3 transformations, albeit all linear. – nan Dec 29 '20 at 4:37
• If you divided by 255 twice, then it's not consistent with the description provided in the documentation. Therefore, I wonder if it did, actually, work -- my assumption is that following the documentation would provide a superior result. But I don't know what the class albumentations.Normalize does; it's not a part of pytorch. Since your question has changed (divide 1 vs 2 times), but this already has an answer, you could ask a new question. – Sycorax Dec 29 '20 at 4:51
• Silly question: Normalizing a collection of values to a given mean and std gives the exact same result regardless of prior scaling. So scaling to [0,1] should have no effect mathematically. Is it maybe all a question of data types (float vs int) or something? – FaultyBagnose May 5 at 6:29
• @FaultyBagnose I don’t understand what you’re asking. Scaling to a unit interval is different than scaling to have 0 mean and unit variance. If you have a new question, you can ask it using the ASK QUESTION button at the top of the page. – Sycorax May 5 at 6:44