7
$\begingroup$

Can anyone explain to me where the formula of skewness or kurtosis comes from? (I mean its derivation.) What's the logic behind it? Who proved it?

$\endgroup$
  • 7
    $\begingroup$ In what sense would someone 'prove' a measure of something? It would be like trying to 'prove' the mean. It's not clear what that could mean. Please clarify. Also note that there's more than one measure of these notions. Are you after the measures based on third and fourth moments? $\endgroup$ – Glen_b -Reinstate Monica Aug 12 '14 at 16:21
  • 2
    $\begingroup$ In addition to @Glen_b's points, be aware that there have been several formulas for skewness, eg. The most common one in textbooks, I'd think, is the 3rd moment. $\endgroup$ – gung - Reinstate Monica Aug 12 '14 at 16:30
  • 3
    $\begingroup$ @gung The third moment is rarely equated to the skewness, which is typically a third moment for standardized data or an estimator thereof. $\endgroup$ – whuber Aug 12 '14 at 17:13
  • $\begingroup$ @whuber, you're right. I should have said 3rd standardized moment. $\endgroup$ – gung - Reinstate Monica Aug 12 '14 at 17:21
  • $\begingroup$ Thank You Very Much All of Your for all your wonderful answers and explanation, and Yeh I meant the third moment. What i mean to say is that why the Skewness is defined as Third central moment, and not Fifth or any other number? Whats the logic behind it. The only explanation i found out was that since Mean is center of the data and raising it to the odd power would cancel the term and final answer will be Zero if it data distribution is symmetric and not zero if it is not symmetric. IS this true? $\endgroup$ – user53932 Aug 18 '14 at 17:39
17
$\begingroup$

You should not expect proof, since skewness and kurtosis are somewhat vague notions.

Symmetry is mathematically precise, but skewness by contrast is surprisingly slippery. Kurtosis is perhaps even more so.

There have been a fairly large number of attempts to give measures corresponding to these notions, but these often-useful measures can be surprisingly counter-intuitive at times. For example, the moment-based skewness can be zero when the distribution is asymmetric (contradicting an assertion one can surprisingly often find when reading elementary texts which discuss skewness).

Can anyone explain to me where the formula of skewness or kurtosis comes from?

Both skewness and kurtosis are somewhat vague terms with several different measures.

These days people mostly mean the moment-based measures, based on standardized third and fourth moments respectively.


Some history

The term "skewness" as applied to a probability distribution seems from an initial look to originate with Karl Pearson, 1895$^{\text{[1]}}$. He begins by talking about asymmetry.

The term "kurtosis" as applied to a probability distribution seems to also originate with Karl Pearson, 1905$^{\text{[2]}}$.

Pearson has formulas for the moment-kurtosis and the square of the moment skewness ($\beta_2$ and $\beta_1$) in his 1895 paper, and they're being used in some sense to help describe shape, even though the notion of kurtosis is not particularly developed there.

However, the idea that higher (standardized) moments than the second can be thought of as some measure of shape or at least of deviation from normality appears to be older than this.

Note, in particular, the historical information contained in Nick Cox's article$^{\text{[3]}}$, here, which makes clear we should give much of the priority to Thiele (1889)$^{\text{[4]}}$.

However, there are other measures than the moment-based quantities. For example, in the case of skewness, there's Pearson's first and second skewness coefficients, which are based on the simple notion of scaling the difference between the mean-and-mode and the mean-and-median respectively. (I think these also date to the 1895 paper but I haven't checked this.)

These different measures relating to the same underlying notion can suggest quite different things (even be opposite in sign). e.g. see here. A reason to be cautious about over-interpreting the moment-kurtosis can be seen here


Edit: Additional information --

It seems that the entries on skewness and kurtosis in Earliest Known Uses of Some of the Words of Mathematics, by Miller et al.$^{[5]}$ agrees with me that it seems they originate with Karl Pearson in 1895 and 1905 respectively.

Nice to have some degree of confirmation.


What I mean to say is that why the skewness is defined as third central moment, and not fifth or any other number? What's the logic behind it?

Okay, so we're specifically dealing with moment skewness.

First, why the third moment makes some sense. Let's begin by thinking of skewness in a somewhat intuitive way rather than rely on a formal definition and see what it might imply.

Recall that $\sigma$ represents a kind of "typical" distance of observations from the mean, and consider what happens when we take a symmetric-looking distribution and make it more right-skew (while trying to keep the area, $\mu$ and $\sigma$ constant):

Right skewness

Here we divide up the axis into four regions - roughly placed at more than one standard deviation below the mean, less than a standard deviation below the mean, less than a standard deviation above the mean and more than a standard deviation above the mean - sections $A$ to $D$ respectively.

For a slight increase in skewness, we tend to see relatively more of the probability immediately to the left of the mean (region $B$) and far above the mean (region $D$), while seeing less immediately above the mean (region $C$) and far below (region $A$).

Indeed, by trying to keep the area constant, if we lift the far tail (the right end of $D$), we're forced to have less probability elsewhere. But if we want to keep the mean constant we must have more probability somewhere lower down. If we put the compensating probability into "$A$" (lowering $B$ and $C$) we could keep the area and mean constant, but we'd end up with a symmetric change (effectively, we would be increasing the variance, not making it more skew).

We can't lift $C$ because lifting both $C$ and $D$ would shift the mean up.

So if we lift the right tail ($D$), while holding the area, mean and standard deviation constant, we can lift the probability in $B$ as well, and reduce the other two -- if we get the relative amounts and positions right within those regions. Keeping $\sigma$ nearly constant constrains what we do more than I have really described (the diagram is slightly inaccurate and as drawn suggests an increase in $\sigma$).

So to make it look a little more right skew, we would tend to shift as described in roughly those areas.

But what does that imply if we try to construct a simple moment-based measure? Note that with third central moments, more area in $A$ or $B$ will tend to reduce it while more area in $C$ or $D$ would tend to increase it (other things being equal), but as we saw, we can't add area without taking it away somewhere else. Cubing things above 1 pulls them out more than cubing things below 1 can "pull them in", so if we add to $D$ while taking away from $C$, the third moment will still increase. Similarly if we add to $B$ while taking away from $A$, the third moment will again tend to increase. That is, the rough-sense "increase in skewness" we just arrived at seems to correspond quite nicely to the third moment.

Now this discussion doesn't rule out fifth and higher moments at all - indeed an increase in third moment will also tend to increase the fifth (unless you do it very carefully indeed), but the (standardized) third moment will be about the simplest way of capturing the notion of skewness using a moment-based measure; while fifth moments are more complex and can move in ways that don't capture our sense of skewness as well as the third moment does.

The third standardized moment doesn't correspond perfectly to our sense of skewness, but it's a pretty good, simple measure that does mostly correspond to it.

The only explanation I found out was that since the mean is center of the data and raising it to the odd power would cancel the term and final answer will be Zero if it data distribution is symmetric and not zero if it is not symmetric. Is this true?

1) In spite of such comments being very easy to find in elementary treatments**, strictly speaking, in respect of probability distributions, neither part is actually true.

a) It's possible for a distribution to be symmetric but not have zero third moment. A simple counterexample is any t-distribution with 3 or fewer degrees of freedom. In samples, however, symmetry implies zero third moment - but samples are almost never perfectly symmetric, so it's not much use there either.

b) It's possible for an asymmetric distribution to have zero third moment.

So symmetry doesn't necessarily imply zero third moment and zero third moment doesn't necessarily imply symmetry.

2) In any case, that wouldn't explain "why third moment rather than fifth", since the fifth power would be just as odd as the third.

** (indeed since I am often asked whether I'd recommend a particular book, the second part (i.e. a claim that $\gamma_1=0$ implies symmetry) is one of several 'quick tests' I use when evaluating whether an elementary book is worth examining more closely -- if a text stumbles over two or more of the common basic errors I tend to see, I don't bother wasting further time looking.)


References

[1]: Pearson, K. (1895),
"Contributions to the Mathematical Theory of Evolution, II: Skew Variation in Homogeneous Material,"
Philosophical Transactions of the Royal Society, Series A, 186, 343-414
[Out of copyright. Freely available here]

[2]: Pearson, K. (1905),
"Das Fehlergesetz und Seine Verallgemeinerungen Durch Fechner und Pearson.", a rejoinder (Skew variation, a rejoinder),
Biometrika, 4 (1905), pp. 169–212.
[While this is also well out of copyright (I can find copies of a Biometrika from 3 years later, for example), I can't locate a copy of this I can link you to. Oxford Journals wants to charge $38 for one day of access to something that is long out of copyright. If you don't have institutional access to one of the places that supply access to it you may be out of luck on this one.]

[3]: Cox, N. J. (2010),
"Speaking Stata: The limits of sample skewness and kurtosis",
The Stata Journal, 10, Number 3, pp. 482–495 (available online here)

[4]: Thiele, T. N. (1889),
Forlæsinger over Almindelig Iagttagelseslære: Sandsynlighedsregning og Mindste Kvadraters Methode,
Copenhagen: C. A. Reitzel. [Out of copyright. There's an English translation - see the Thiele reference in [3].]

[5]: Miller, Jeff; et al., (Accessed 16 August 2014),
Earliest Known Uses of Some of the Words of Mathematics,
See here

$\endgroup$
  • 4
    $\begingroup$ Various historical comments are sprinkled throughout my paper, Cox, N.J. 2010. The limits of sample skewness and kurtosis. Stata Journal 10: 482-495 accessible to all at stata-journal.com/sjpdf.html?articlenum=st0204 @Glen_b is correct that there is a history before Karl Pearson, who was less than gracious, and less than candid, in acknowledging the prior work of T.N. Thiele. $\endgroup$ – Nick Cox Aug 13 '14 at 8:17
  • $\begingroup$ Biometrika is accessible via www.jstor.org if you have access to that. $\endgroup$ – Nick Cox Aug 13 '14 at 8:18
  • 4
    $\begingroup$ (mean $-$ median) / SD has the interesting and at first sight surprising property that it lies in $[-1,1]$, something periodically rediscovered. Although necessarily no single measure can capture all that might be desired, it deserves a little more use as a descriptive measure. It will occasionally have the "wrong" sign, as will any such measure. L-moments deserve more use for describing distribution shape. $\endgroup$ – Nick Cox Aug 13 '14 at 8:23
  • $\begingroup$ @Nick - thanks especially for the mention of Thiele; your article is well worth reading -- indeed I will put a link in my answer. I had occasion to consult Lauritzen's work many years ago when I was writing a small item in part about Thiele's work on a (slightly) more basic form of what we now call the Kalman filter, but I could only have the book for a couple of hours and focused only on the sections I needed; I obviously missed much of importance. Thiele was - and continues to be - terribly under-appreciated. $\endgroup$ – Glen_b -Reinstate Monica Aug 13 '14 at 8:50
  • 2
    $\begingroup$ Indeed: Lauritzen's book on Thiele contains fascinating material. See ukcatalogue.oup.com/product/9780198509721.do but be warned that it is expensive. $\endgroup$ – Nick Cox Aug 13 '14 at 9:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.