Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 5 down vote favorite

I'm a PhD student and doing a research on regression analysis.

My question is how to determine whether the data is slightly, moderately or extremely non-normally distributed?

share|improve this question
You could plot the histogram using "hist" in R, and see if it follows the bell shape of the normal. Alternatively, you could also try the qqplot in R, which plots the quantiles of the normal against the quantiles of your data – JCWong Aug 30 '12 at 5:51
4  
qqnorm() in R does what qqplot does, but you don't have to supply the other distribution. Also, you may want to check out stats.stackexchange.com/questions/2492/… – naught101 Aug 30 '12 at 5:51
3  
Words like "slightly," "moderately," and "extremely" are too vague to make a definitively answerable question. Could you please tell us more about what you need to know? – whuber♦ Aug 30 '12 at 11:12

5 Answers

up vote 3 down vote

The sample skewness $$\gamma=\frac{\sum_{i=1}^n(x_i-\bar{x})^3}{\Big(\sum_{i=1}^n(x_i-\bar{x})^2\Big)^{3/2}}$$ and the sample (excess) kurtosis $$\kappa=\frac{\sum_{i=1}^n(x_i-\bar{x})^4}{\Big(\sum_{i=1}^n(x_i-\bar{x})^2\Big)^{2}}-3$$ are often used as measures of non-normality.

The sample skewness measures the asymmetry of the empirical distribution. If it is far from $0$, the distribution is not very symmetric. Since the normal distribution is symmetric, a sample from the normal distribution should be close to $0$.

The sample kurtosis measures the "peakedness" of the distribution. If it is much greater than $0$, then the distribution is more peaked than the normal distribution, which typically means that it has heavier tails. If it is less than $0$ it is less peaked, which typically means that the distribution is bimodal. The sample kurtosis is bounded from below by $-2$ (a value that is obtained for a two-point distribution, which of course is extremely bimodal).!

Here are two examples (normal distribution in grey, other distributions in red):

enter image description here

The skew distribution has theoretical skewness $1.6$ whereas the kurtotic distributions has theoretical (excess) kurtosis $1.5$. As you can see, the kurtotic distribution has heavier tails than the normal distribution.

So, why use skewness and kurtosis as quantifications of non-normality? The main reason is that they affect the asymptotics of the central limit theorem, which as you may know often can be used to motivate the use of a statistical procedure (that is based on normality) even if the data does not come from a normal distribution, given that you have a "large enough" sample. If either the skewness or the kurtosis is high, larger sample sizes are needed for such motivations to be valid.

For some inferential procedures you need to worry more about skewness, and for some you need to worry about heavy tails (kurtosis). I've written more about that elsewhere on this site.

share|improve this answer
up vote 3 down vote

Although a wikipedia link might be not considered extremely helpful, the list of methods for testing normality is quite long.

The methods range from (already mentioned) histograms and qq-Plots if you want to stay on the graphical side over more lets say "empirical tests" (multiple sigma events in relation to sample size) to parametric and nonparametric statistical tests. A complete review of these would be, in my opinion, out of scope here (and for some methods definitely also out of my scope) so I will be quite frank here.

Since you were mentioning regression analysis I guess you want to test the normality of the residuals. Just use one of the normality tests on the wiki page. The more popular variants compare skewness and kurtosis to that of a normal distribution. The nonparametric versions are Kolmogorov-Smirnov type of tests that use the empirical cumulative distribution function of your data (probably the residuals). Just look at the wiki page. The standard tests are quite simple to implement.

share|improve this answer
3  
The tests can be useful, but be careful because the p-values will be partly dependent on sample size. – Peter Flom Aug 30 '12 at 10:18
I don't think he wants to know if his residuals are normally distributed (Surely there's some difference from a normal distribution); he wants a way to quantify divergence from normality. – Stumpy Joe Pete Sep 14 '12 at 19:06
up vote 3 down vote

I think the other answers really address methods for deciding non-normality. But I think the OP asks a different question. Basically he is asking about how to decide once normality is rejected how do you determine the severity? If it is mild perhaps the deviation can be ignored. Skewness and kurtosis can be looked at as measures of non-normality but I think it comes down to a subjective decision as to how large a difference should be to call it, mild, moderate or large. I think the point is that this decision can be made by looking at histograms, qq plots, or the magnitude of the skewness and kutosis. But this is going to be subjective and not formal.

share|improve this answer
What about calculating the relative entropy between the sample distribution and a hypothetical best fitting normal? – John Sep 14 '12 at 22:06
1  
Another measure but still picking a number to define severity will be subjective. – Michael Chernick Sep 14 '12 at 22:09
1  
+1 This answer is on the right track. It seems a little severe to suggest the evaluation of non-normality is subjective, though. The question of "amount" of non-normality bears on the consequences of departures from normality: in some applications there is little consequence--consider robust tests--and in others the consequences can be large. By considering these consequences explicitly, and the effects of non-normality on them, one can render an assessment that is more "objective"--or at least more useful and defensible. – whuber♦ Sep 14 '12 at 22:42
@whuber I was not really saying that. I think nonnormality can be assessed using goodness of fit tests keeping in mind that sample size should not be excessively large. I agree with you that severity depends on the robustness of the procedure. But even so at some point you look at the nonnormality see the consequences and ask yourself whether or not to live with it or try something else. I think that is a very subjective but sometimes a real decision that people will make. – Michael Chernick Sep 14 '12 at 22:58
up vote 1 down vote

The Skewness of the Normal distribution is 0. The Kurtosis of the Normal distribution is 0. Those two statistics are concrete measures of distribution characteristics as opposed to subjective plot interpretation. Of course the question remaining is how far from Normal your distribution is for $n$ points and some value of Kurtosis and Skewness.

Hence you will be better off by running some Normality Tests. The Shapiro-Wilk is a sensible choice for univariate data.

If you use R you will find the function shapiro.test() in the stats package useful. The moments package includes the functions skewness() and kurtosis() plus jarque.test() if you want a second opinion after the results of the Shapiro-Wilk test.

share|improve this answer
4  
You should speak of excess kurtosis being $0$ or kurtosis being $3$. Just to be sure. – Richard Aug 30 '12 at 9:45
Good point @Richard; the kurtosis definition is given both ways (and some software subtracts 3 and some doesn't) – Peter Flom Aug 30 '12 at 10:16
@PeterFlom I know my comment was a bit fussy but I have seen so many people mixing things up with kurtosis (e.g. in the Cornish Fisher expansion) so that I started stressing "excess" or "not excess". – Richard Aug 30 '12 at 11:58
You're right - it's a confusing state of affairs – Peter Flom Aug 30 '12 at 13:36
up vote 0 down vote

TQ to all responses of my question. But, may be my question is not so clear. Ok, let say i have some different values of skewness and kurtosis (for example: skewness = 1.5, kurtosis = 2.0). So my question is, from the values of skewness and kurtosis, what is the type of data distribution? is it moderately non-normal, or extremely non-normal or what?

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.