Why do we need to normalize the images before we put them into CNN?

I am not clear the reason that we normalise the image for CNN by (image - mean_image)? Thanks!

First note: you really should be also dividing by the standard deviation of each feature (pixel) value as well. Subtracting the mean centers the input to 0, and dividing by the standard deviation makes any scaled feature value the number of standard deviations away from the mean.

To answer your question: Consider how a neural network learns its weights. C(NN)s learn by continually adding gradient error vectors (multiplied by a learning rate) computed from backpropagation to various weight matrices throughout the network as training examples are passed through.

The thing to notice here is the "multiplied by a learning rate".

If we didn't scale our input training vectors, the ranges of our distributions of feature values would likely be different for each feature, and thus the learning rate would cause corrections in each dimension that would differ (proportionally speaking) from one another. We might be over compensating a correction in one weight dimension while undercompensating in another.

This is non-ideal as we might find ourselves in a oscillating (unable to center onto a better maxima in cost(weights) space) state or in a slow moving (traveling too slow to get to a better maxima) state.

It is of course possible to have a per-weight learning rate, but it's yet more hyperparameters to introduce into an already complicated network that we'd also have to optimize to find. Generally learning rates are scalars.

Thus we try to normalize images before using them as input into NN (or any gradient based) algorithm.

• what about color images? do i do it for each color channel? wouldn't that mess up the color distribution? – user10024395 Jun 28 '16 at 0:19
• Believe you're supposed to yes. You can just do something like: (image - image.mean()) / (image.std() + 1e-8) – JohnAllen Sep 23 '16 at 6:07
• Is this really helpful for training? I am training object detector, and using this fitlers, the image(s) are really confusing, its hard to see objects that i am detecting – Darlyn Apr 24 '18 at 19:36
• It’s quite helpful for training in terms of learnability and accuracy - it’s not for you it’s for the model :) You might want to output the non-normalized image when you’re debugging so that it appears normal to your human eyes. – lollercoaster Apr 24 '18 at 20:21