4
$\begingroup$

Wikipedia says:

For the second and higher moments, the central moments (moments about the mean, with c being the mean) are usually used rather than the moments about zero, because they provide clearer information about the distribution's shape.

Could someone explain/convince me why this is true? Why is there a discrepancy?
This has always bugged me and I have never seen a good explanation for it -- I just don't quite understand why/how standardization provides "clear" information in one case but not in another.

For example:

  1. To compute the skewness, why not standardize both the mean and the variance?
  2. To compute the kurtosis, why not standardize the mean, the variance, and the skewness?
  3. ...
  4. To compute the nth moment, why not first standardize all the mth moments for m < n?
    If standardization is useful then why only do this for m = 1?
$\endgroup$
11
  • 1
    $\begingroup$ How do you understand "shape"? I take it to be the collection of all properties of a distribution that are not altered by any change of location or scale--in other words, properties that persist in a graph of the distribution when all axis labels are erased. If you share this understanding then (a) the answer to your question should become obvious and (b) it will be evident that central moments are not the only way to solve the problem of describing shapes; they are merely one way to establish a location and scale for (most) distributions. $\endgroup$
    – whuber
    Commented Oct 27, 2014 at 14:41
  • 1
    $\begingroup$ The word "normalize" is one of many in statistical science that changes meaning from field to field, to the extent that it is dangerous. Using it to imply "mean-subtracted" isn't standard for many of us. I would be exceeding my knowledge to say that it is non-standard for all, but I challenge you to cite literature where "normalize" is identical to "subtract the mean". $\endgroup$
    – Nick Cox
    Commented Oct 27, 2014 at 19:00
  • $\begingroup$ "The second type of normalization originates from statistics, and eliminates the unit of measurement by transforming the data into new scores with a mean of 0 and a standard deviation of 1." @NickCox I think my usage of the word wasn't too outlandish and made enough sense to get the point across, so let's not go on a tangent here. $\endgroup$
    – user541686
    Commented Oct 27, 2014 at 19:32
  • $\begingroup$ Sorry; that's not what I asked. Your question was why use moments about the mean rather than moments about zero. For example, the second moment about the mean is the variance; it's not scaled by the standard deviation. Naturally I agree that skewness and kurtosis are often defined as moment ratios, which is equivalent to scaling by the standard deviation too, but neither is mentioned in your question at all. In short, my comment is about the wording in your question. You've provided evidence for my assertion, as subtracting mean and dividing by SD is commonly called standardization. $\endgroup$
    – Nick Cox
    Commented Oct 27, 2014 at 19:39
  • 1
    $\begingroup$ I didn't say that I felt confused; unfortunately I remain of the view that the precise import of your question is likely to be unclear to others. A paper with tutorial flavour at stata-journal.com/sjpdf.html?articlenum=st0204 may have interest for people curious about moments. $\endgroup$
    – Nick Cox
    Commented Oct 27, 2014 at 20:36

1 Answer 1

4
$\begingroup$

Since the question was updated, I update my answer:

The first part (To compute the skewness, why not standardize both the mean and the variance?) is easy: That is precisely how it's done! See the definitions of skewness and kurtosis in wiki.

The second part is both easy and hard. On one hand we could say that it is impossible to normalize random variable to satisfy three moment conditions, as linear transformation $X \to aX + b$ allows only for two. But on the other hand, why should we limit ourselves to linear transformations? Sure, shift and scale are by far the most prominent (maybe because they are sufficient most of the time, say for limit theorems), but what about higher order polynomials or taking logs, or convolving with itself? In fact, isn't it what Box-Cox transform is all about -- removing skew?

But in the case of more complicated transformations, I think, the context and the transformation itself becomes important, so maybe that is why there are no more "moments with names". That does not mean that r.v.s are not transformed and that the moments are not calculated, on the contrary. You just chose your transformation, calculate what you need and move on.


The old answer about why centralized moments represent shape better than raw:

The keyword is shape. As whuber suggested, by shape we want consider the properties of the distribution that are invariant to translation and scaling. That is, when you consider variable $X + c$ instead of $X$, you get the same distribution function (just shifted to the right or left), so we would like to say that its shape stayed the same.

The raw moments do change when you translate the variable, so they reflect not only the shape, but also a location. In fact, you can take any random variable, and shift it $X \to X + c$ appropriately to get any value for its, say, raw third moment.

The same observation holds for all odd moments and to lesser extent for even moments (they are bounded from below and lower bound does depend on shape).

The centralized moment, on the other hand, does not change when you translate the variable, so that's why they are more descriptive of the shape. For example, if your even centralized moment is large, you known that random variable has some mass not too close to mean. Or if your odd moment is zero, you known that your random variable has some symmetry around mean.

The same argument extends to scale, which is transformation $X\to cX$. The usual normalization in this case is division by standard deviation, and the corresponding moments are called normalized moments, at least by wikipedia.

$\endgroup$
12
  • $\begingroup$ Could you explain your assertion about "move it around to get any value of third moment"? What exactly do you mean by "move it around," what bearing does this operation have on distributional shape, and why does it change the third moment? $\endgroup$
    – whuber
    Commented Oct 27, 2014 at 14:43
  • $\begingroup$ Sure: by moving around I meant translations $X \to X + c$. It does obviously change the value of the third moment and you can get it be equal to whatever value. It does not change the shape of the distribution by your nice definition of shape above. $\endgroup$
    – psarka
    Commented Oct 27, 2014 at 15:25
  • $\begingroup$ Ah... you mean the raw third moment rather than the central third moment. In this context, where we are discussing several kinds of moments, I lost track of which one you actually meant. That misreading was surely my fault, but when you modify this post to clarify what "move it around means" you might consider making some additional minor edits to help prevent others from falling into the same trap. $\endgroup$
    – whuber
    Commented Oct 27, 2014 at 15:31
  • $\begingroup$ (+1) Many thanks for turning this into a really clear, authoritative post. $\endgroup$
    – whuber
    Commented Oct 27, 2014 at 16:32
  • 1
    $\begingroup$ Aaahh! Now I get it. The question is: why don't we normalize by requiring, say, that third moment was equal to zero, and that the tenth one was equal to one? OK, that's a completely different question, let me think about it :) $\endgroup$
    – psarka
    Commented Oct 27, 2014 at 19:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.