I am confused whether the standardization (subtract mean and divide by std) should be done per image basic or across the overall dataset. While overall dataset makes more sense, popular libraries like TensorFlow provide functions like tf.image.per_image_standardization that does the following.

Linearly scales image to have zero mean and unit norm.

This op computes (x - mean) / adjusted_stddev, where mean is the average of all values in image, and adjusted_stddev = max(stddev, 1.0/sqrt(image.NumElements())).

stddev is the standard deviation of all values in image. It is capped away from zero to protect against division by 0 when handling uniform images.

Is this good enough?


1 Answer 1


Each method has their own purposes. In sequential data such as speech [1], the mean and covariance are calculated from an utterance (a recording) and is then subtracted from all the observations in that utterance. This is done for each utterance separately.

In images on the other hand, one image can be seen as a sequence of pixels. Therefore the mean and variance in an image are calculated from individual pixels in that image. For pixed-wise or per-image normalization, mean and covariance are calculated for each image separately.

In case of the overall normalization, it is better though to calculate the mean and variance from the training data and use it to normalize all the sets including training, validation, test etc.


  • 5
    $\begingroup$ The last sentence is a key. If you normalize over all of your data -- training, testing, etc -- you are leaking information into your training set that you otherwise wouldn't have. In the real world, you won't be able to normalize over all of your future data points. $\endgroup$
    – Wayne
    Jan 12, 2018 at 15:12
  • $\begingroup$ Thanks fo the answer. Further, how important do you think are each are? How often each method is used? Can performing both hamper the performance? $\endgroup$ Jan 12, 2018 at 16:26
  • $\begingroup$ I think the latter (overall) is more used and more common. The other (per image) might come in handy for amplitude normalization perhaps. But one should try and see the difference. $\endgroup$
    – PickleRick
    Jan 12, 2018 at 16:29
  • 1
    $\begingroup$ I have tried this and got lower accuracy when using per image normalization compared to subtracting the mean of the data-set from each example $\endgroup$
    – Kasparov92
    Mar 17, 2018 at 7:31
  • 1
    $\begingroup$ @Wayne upvoted your comment because its also reasonable, but the real reason why you use train mean and variance for both train and test sets is to avoid "domain shift" (occurs when test set and train set distributions no longer overlap). This is the simplest form of what is called "domain adaptation." $\endgroup$ Sep 3, 2019 at 0:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.