How to determine whether data is slightly or extremely non-normally distributed?

I'm a PhD student and doing a research on regression analysis.

My question is how to determine whether the data is slightly, moderately or extremely non-normally distributed?

TQ to all responses of my question. But, may be my question is not so clear. Ok, let say i have some different values of skewness and kurtosis (for example: skewness = 1.5, kurtosis = 2.0). So my question is, from the values of skewness and kurtosis, what is the type of data distribution? is it moderately non-normal, or extremely non-normal or what?

-
You could plot the histogram using "hist" in R, and see if it follows the bell shape of the normal. Alternatively, you could also try the qqplot in R, which plots the quantiles of the normal against the quantiles of your data –  JCWong Aug 30 '12 at 5:51
qqnorm() in R does what qqplot does, but you don't have to supply the other distribution. Also, you may want to check out stats.stackexchange.com/questions/2492/… –  naught101 Aug 30 '12 at 5:51
Words like "slightly," "moderately," and "extremely" are too vague to make a definitively answerable question. Could you please tell us more about what you need to know? –  whuber Aug 30 '12 at 11:12
@Nor 1) what does TQ mean and 2) The distribution will not be uniquely identified by its skewness and kurtosis. The third and fourth moments alone do not characterize a distribution. Regarding its relationship to normality, specifically, I think the existing answers address this. –  Macro Aug 9 at 18:34
"from the values of skewness and kurtosis, what is the type of data distribution?" --- you cannot determine the distribution from the sample skewness and kurtosis. Indeed, you can't even do it from the population skewness and kurtosis. –  Glen_b Aug 9 at 23:53

The sample skewness $$\gamma=\frac{\sum_{i=1}^n(x_i-\bar{x})^3}{\Big(\sum_{i=1}^n(x_i-\bar{x})^2\Big)^{3/2}}$$ and the sample (excess) kurtosis $$\kappa=\frac{\sum_{i=1}^n(x_i-\bar{x})^4}{\Big(\sum_{i=1}^n(x_i-\bar{x})^2\Big)^{2}}-3$$ are often used as measures of non-normality.

The sample skewness measures the asymmetry of the empirical distribution. If it is far from $0$, the distribution is not very symmetric. Since the normal distribution is symmetric, a sample from the normal distribution should be close to $0$.

The sample kurtosis measures the "peakedness" of the distribution. If it is much greater than $0$, then the distribution is more peaked than the normal distribution, which typically means that it has heavier tails. If it is less than $0$ it is less peaked, which typically means that the distribution is bimodal. The sample kurtosis is bounded from below by $-2$ (a value that is obtained for a two-point distribution, which of course is extremely bimodal).!

Here are two examples (normal distribution in grey, other distributions in red):

The skew distribution has theoretical skewness $1.6$ whereas the kurtotic distributions has theoretical (excess) kurtosis $1.5$. As you can see, the kurtotic distribution has heavier tails than the normal distribution.

So, why use skewness and kurtosis as quantifications of non-normality? The main reason is that they affect the asymptotics of the central limit theorem, which as you may know often can be used to motivate the use of a statistical procedure (that is based on normality) even if the data does not come from a normal distribution, given that you have a "large enough" sample. If either the skewness or the kurtosis is high, larger sample sizes are needed for such motivations to be valid.

For some inferential procedures you need to worry more about skewness, and for some you need to worry about heavy tails (kurtosis). I've written more about that elsewhere on this site.

-
Though beware - a test of skewness and kurtosis will be all but powerless to detect quite substantial deviations from normality, such as in the illustration here –  Glen_b Aug 10 at 0:05
To clarify: skewness/kurtosis tests often have very favourable power properties, but there are examples, such as the one given by @Glen_b (a very nice example, at that!), where they fail miserably. I wonder how common such distributions are in practice though... –  MånsT Aug 13 at 9:38
well, bimodal distributions do occur in practice, but I expect that in most cases of interest at least one of skewness or kurtosis will tend to be different. It's important to be aware that there are distinctly non-normal distributions that are not picked up by testing the 3rd or 4th cumulant. Interestingly, there are asymmetric distributions whose third moment-skewness is zero. –  Glen_b Aug 13 at 9:55

I think the other answers really address methods for deciding non-normality. But I think the OP asks a different question. Basically he is asking about how to decide once normality is rejected how do you determine the severity? If it is mild perhaps the deviation can be ignored. Skewness and kurtosis can be looked at as measures of non-normality but I think it comes down to a subjective decision as to how large a difference should be to call it, mild, moderate or large. I think the point is that this decision can be made by looking at histograms, qq plots, or the magnitude of the skewness and kutosis. But this is going to be subjective and not formal.

-
What about calculating the relative entropy between the sample distribution and a hypothetical best fitting normal? –  John Sep 14 '12 at 22:06
Another measure but still picking a number to define severity will be subjective. –  Michael Chernick Sep 14 '12 at 22:09
+1 This answer is on the right track. It seems a little severe to suggest the evaluation of non-normality is subjective, though. The question of "amount" of non-normality bears on the consequences of departures from normality: in some applications there is little consequence--consider robust tests--and in others the consequences can be large. By considering these consequences explicitly, and the effects of non-normality on them, one can render an assessment that is more "objective"--or at least more useful and defensible. –  whuber Sep 14 '12 at 22:42
@whuber I was not really saying that. I think nonnormality can be assessed using goodness of fit tests keeping in mind that sample size should not be excessively large. I agree with you that severity depends on the robustness of the procedure. But even so at some point you look at the nonnormality see the consequences and ask yourself whether or not to live with it or try something else. I think that is a very subjective but sometimes a real decision that people will make. –  Michael Chernick Sep 14 '12 at 22:58
How about the distance measure used in Lillifor's test related to the Kolmogorov-Smirnov test? But why do we care about this so much as opposed to using robust methods such as semiparametric models? –  Frank Harrell Aug 9 at 21:57

Although a wikipedia link might be not considered extremely helpful, the list of methods for testing normality is quite long.

The methods range from (already mentioned) histograms and qq-Plots if you want to stay on the graphical side over more lets say "empirical tests" (multiple sigma events in relation to sample size) to parametric and nonparametric statistical tests. A complete review of these would be, in my opinion, out of scope here (and for some methods definitely also out of my scope) so I will be quite frank here.

Since you were mentioning regression analysis I guess you want to test the normality of the residuals. Just use one of the normality tests on the wiki page. The more popular variants compare skewness and kurtosis to that of a normal distribution. The nonparametric versions are Kolmogorov-Smirnov type of tests that use the empirical cumulative distribution function of your data (probably the residuals). Just look at the wiki page. The standard tests are quite simple to implement.

-
The tests can be useful, but be careful because the p-values will be partly dependent on sample size. –  Peter Flom Aug 30 '12 at 10:18
I don't think he wants to know if his residuals are normally distributed (Surely there's some difference from a normal distribution); he wants a way to quantify divergence from normality. –  Stumpy Joe Pete Sep 14 '12 at 19:06

The Skewness of the Normal distribution is 0. The Kurtosis of the Normal distribution is 0. Those two statistics are concrete measures of distribution characteristics as opposed to subjective plot interpretation. Of course the question remaining is how far from Normal your distribution is for $n$ points and some value of Kurtosis and Skewness.

Hence you will be better off by running some Normality Tests. The Shapiro-Wilk is a sensible choice for univariate data.

If you use R you will find the function shapiro.test() in the stats package useful. The moments package includes the functions skewness() and kurtosis() plus jarque.test() if you want a second opinion after the results of the Shapiro-Wilk test.

-
You should speak of excess kurtosis being $0$ or kurtosis being $3$. Just to be sure. –  Richard Aug 30 '12 at 9:45
Good point @Richard; the kurtosis definition is given both ways (and some software subtracts 3 and some doesn't) –  Peter Flom Aug 30 '12 at 10:16
@PeterFlom I know my comment was a bit fussy but I have seen so many people mixing things up with kurtosis (e.g. in the Cornish Fisher expansion) so that I started stressing "excess" or "not excess". –  Richard Aug 30 '12 at 11:58
You're right - it's a confusing state of affairs –  Peter Flom Aug 30 '12 at 13:36