For centering and scaling a variable (e.g. prior to a regression, or to a visualization), the standard procedure, of course, is to subtract the mean then divide by the standard deviation.

But is it considered preferable to use the population standard deviation (i.e. divided by n) or the sample standard deviation (divided by n-1)? Does it depend on one's use?

Interestingly, the standard R and Python functions seem to make different choices here. Python's sklearn.preprocessing.scale() uses population standard deviation; R's scale() uses sample standard deviation.

(NOTE: there's a prior question here, but it pertains to a very specific psychological method, and the one answer isn't actually substantiated by anything.)

  • 1
    $\begingroup$ We should not mix population variance (or st. dev.) - with df=N, with sample's variance, with df=n. Neither are an estimate of the parameter: the 1st is parameter, the 2nd is pure statistic. We may use the sample's variance as the estimate of population one, but it is biased; it is called maximum likelihood estimate of variance. $\endgroup$ – ttnphns Nov 28 '16 at 8:16
  • $\begingroup$ (cont.) A better option is to compute what is nicknamed "sample variance", which is on df=n-1 and is unbiased estimate of population variance. That "sample" or unbiased estimate one shouldn't be confused with the above sample's variance. $\endgroup$ – ttnphns Nov 28 '16 at 8:16
  • $\begingroup$ The use of n versus n-1 has to do with whether to choose a biased vs unbiased estimate of variance & has nothing to do with the population variance. $\endgroup$ – Michael R. Chernick Sep 22 '19 at 14:28

Practically speaking the population variance is usually not known. So you don't have a choice. If the population variance is known and hence also the population standard deviation, then of course it is best to scale by the population standard deviation.

  • $\begingroup$ True enough. Any idea what the thinking might have been behind the scikit-learn implementation then? $\endgroup$ – Paul Gowder Nov 28 '16 at 4:04
  • 3
    $\begingroup$ The meaning of "population standard deviation" in the question is what most people call the "sample standard deviation" while the meaning of "sample standard deviation" in the question would be called an "estimator" of the standard deviation. Neither one is based on the standard deviation of the assumed underlying population, process, or model, which is what is usually meant by "population standard deviation." By confounding these various meanings of "population standard deviation," this answer misses the mark and risks misleading readers. $\endgroup$ – whuber Nov 28 '16 at 16:22
  • 1
    $\begingroup$ @whuber You say that population standard deviation is "what most people call the'sample' standard deviation". Who are these people? I think you are suggesting that most readers don't understand the difference between a sample and the population. Why would anyone assume that the OP doesn't know what h is talking about? Based on his comment at least Paul Gowder knows what I am talking about. Over the past two days we have had a couple of amazing disagreements. I generally respect your judgment. But these comment are blowing me away! $\endgroup$ – Michael R. Chernick Nov 29 '16 at 17:07
  • 2
    $\begingroup$ Michael, please do not interpret any of my comments, ever, as insinuations about what someone might or might not understand. The current question can be understood as distinguishing two formulas for a "standard deviation" insofar as one divides by $n$ and another by $n-1$. Regardless of what they might be called, the question appears to concern which statistic to use. It does not appear to refer to the standard deviation of a distribution used in a model or hypothetical "population"--but that is precisely the sense in which you are answering. Therefore your answer misses the point. $\endgroup$ – whuber Nov 29 '16 at 18:36
  • 4
    $\begingroup$ To put @whuber's point otherwise: In some circumstances, the population standard deviation is considered or declared to be known; this might be, exceptionally, because the entire population happens to be known; or, more commonly, because we are just playing a game and make up the rules (e.g. declare that our population is $N(0, 1)$ and then generate samples in software). But anything based on a sample of size $n$, how I interpret the notation here, cannot be the population standard deviation. It is the SD of the sample. $\endgroup$ – Nick Cox Nov 29 '16 at 18:50

I was wondering the same thing and I tend to think that this should depend on the intended use.

If the reason of standardization is to use the standardized version for further work concerning new samples (such as standardization before a machine learning process), I can understand that the values at hand would be considered as a sample and the standard deviation would be calculated as that of a sample.

However, if you are going to use the sample values at hand for a comparison within themselves without further application of new samples, as in the link you shared:

In this context, as elsewhere, the standard deviation is used to make scores comparable, and no statistical inference to a population is implied.

then I would consider the samples at hand as the population being compared and tend to use a population standard deviation.

This might not be a good example but let us assume that we took a sample of 10 observations out of a population. If, for some reason, I only need to produce a standardized distance comparison among those 10 observations (without any statistical inference to another new sample other than that 10), then those 10 observations become my population for the comparison within temselves.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.