Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

In many different statistical methods there is an "assumption of normality". What is "normality" and how do I know if there is normality?

share|improve this question
1  
did you try google/ wikipedia first ? en.wikipedia.org/wiki/Normal_distribution –  robin girard Jul 25 '10 at 20:13

7 Answers 7

up vote 24 down vote accepted

The assumption of normality is just the supposition that the underlying random variable of interest is distributed normally, or approximately so. Intuitively, normality may be understood as the result of the sum of a large number of independent random events.

More specifically, normal distributions are defined by the following function:

alt text

where $\mu$ and $\sigma^2$ are the mean and the variance, respectively, and which appears as follows:

alt text

This can be checked in multiple ways, that may be more or less suited to your problem by its features, such as the size of n. Basically, they all test for features expected if the distribution were normal (e.g. expected quantile distribution).

share|improve this answer
4  
Do you get bonus points for the first graph in the stats exchange? –  csgillespie Jul 19 '10 at 20:53

One note: The assumption of normality is often NOT about your variables, but about the error, which is estimated by the residuals. For example, in linear regression $Y = a + bx + e$; there is no assumption that $Y$ is normally distributed, only that $e$ is.

share|improve this answer
8  
+1. Finally someone has pointed out what perhaps is the most important aspect of this question: in most situations, "normality" is important in regard to residuals or to sampling distributions of statistics, not in regard to the distributions of the populations! –  whuber Oct 21 '11 at 20:27
3  
I would add that if $e$ is normally distributed, then Y is at least conditionally normal as well. I think its this that gets missed - people think that Y is marginally normal but its actually conditional normality that is needed. The simplest example of this is a one way ANOVA. –  probabilityislogic Oct 6 '12 at 1:34
    
Conditionally on what? –  PeterRabbit Jan 28 at 5:46

A related question can be found here about the normal assumption of the error (or more generally of the data if we do not have prior knowledge about the data).

Basically,

  1. It is mathematically convenient to use normal distribution. (It's related to Least Squares fitting and easy to solve with pseudoinverse)
  2. Due to Central Limit Theorem, we may assume that there are lots of underlying facts affecting the process and the sum of these individual effects will tend to behave like normal distribution. In practice, it seems to be so.

An important note from there is that, as Terence Tao states here, "Roughly speaking, this theorem asserts that if one takes a statistic that is a combination of many independent and randomly fluctuating components, with no one component having a decisive influence on the whole, then that statistic will be approximately distributed according to a law called the normal distribution".

To make this clear, let me write a Python code snippet

# -*- coding: utf-8 -*-
"""
Illustration of the central limit theorem

@author: İsmail Arı, http://ismailari.com
@date: 31.03.2011
"""

import scipy, scipy.stats
import numpy as np
import pylab

#===============================================================
# Uncomment one of the distributions below and observe the result
#===============================================================
x = scipy.linspace(0,10,11)
#y = scipy.stats.binom.pmf(x,10,0.2) # binom
#y = scipy.stats.expon.pdf(x,scale=4) # exp
#y = scipy.stats.gamma.pdf(x,2) # gamma
#y = np.ones(np.size(x)) # uniform
y = scipy.random.random(np.size(x)) # random

y = y / sum(y);

N = 3
ax = pylab.subplot(N+1,1,1)
pylab.plot(x,y)

# Plotting details 
ax.set_xticks([10])
ax.axis([0, 2**N * 10, 0, np.max(y)*1.1])
ax.set_yticks([round(np.max(y),2)])

#===============================================================
# Plots
#===============================================================
for i in np.arange(N)+1:
    y = np.convolve(y,y)
    y = y / sum(y);    

    x = np.linspace(2*np.min(x), 2*np.max(x), len(y))
    ax = pylab.subplot(N+1,1,i+1)
    pylab.plot(x,y)
    ax.axis([0, 2**N * 10, 0, np.max(y)*1.1])
    ax.set_xticks([2**i * 10])
    ax.set_yticks([round(np.max(y),3)])

pylab.show()

Random distribution

Exponential distribution

Uniform distribution

As can be seen from the figures, the resulting distribution (sum) tends towards a normal distribution regardless of the individual distribution types. So, if we do not have enough information about the underlying effects in the data, normality assumption is reasonable.

share|improve this answer
    
The CLT doesn't allow us to assume there are lots of individual effects in any given process - if we are given that there are lots of not-too-dependent individual factors contributing to a measurement (none of which have too much of the total variation), we may be justified in assuming normality by invoking the CLT. The assumption of many contributions precedes the application of the CLT, it's not in any sense a result of the CLT. Otherwise everything would be normal, when in fact that's only sometimes roughly true. –  Glen_b May 21 at 6:01

You can't know whether there normality and that's why you have to make an assumption that's there. You can only prove the absence of normality with statistic tests.

Even worse, when you work with real world data it's almost certain that there isn't true normality in your data.

That means that your statistical test is always a bit biased. The question is whether you can live with it's bias. To do that you have to understand your data and the kind of normality that your statistical tool assumes.

It's the reason why Frequentist tools are as subjective as Bayesian tools. You can't determine based on the data that it's normally distributed. You have to assume normality.

share|improve this answer
3  
You cannot prove anything using statistics. A proof is meant to be exact. Statistics is about probabilities. Even a p=0.99 result of a Chi squared does not "prove" that the underlying distribution is not normal. It's just damn unlikely that it's normal. –  xmjx Jun 16 '11 at 7:53
    
@xmjx: You can't even say that a given distribution is probably normal distributed. If you have a distribution with where 99.99% of your values are 1 but 0.01% of your values are 1000000 a statistic test that samples 100 values has a good chance to tell you wrongly that your distribution is normally distributed. –  Christian Oct 21 '11 at 19:44
1  
I'm not much of a statistical expert, so this may seem like a silly question... doesn't "true normality" exist in the underlying process that generates the variable rather than the data? It may seem like a silly distinction, but perhaps it could save some soul-searching. If the gathered data is not exactly normal, but the underlying random process works in a basically normal way, is that a situation where you could decide to "live with the bias"? –  Jonathan Oct 21 '11 at 21:28
    
@Christian - your comment that "...100 values has a good chance..." isn't borne out at all by my hacking: x=c(rep(1,99),rep(1000000,1)); ks.test(x,pnorm) > The assumption of normality is still "rejected" by the K-S Test. –  rolando2 Oct 5 '12 at 22:27
    
I like this answer (+1) but it is a bit pessimistic about what can be done with the assumption of normality. It is usually a good starting point for any modelling, and you can generalise to a very wide class of distributions by taking either mixtures or functions of normally distributed random variables. –  probabilityislogic Oct 6 '12 at 1:46

The assumption of normality assumes your data is normally distributed (the bell curve, or gaussian distribution). You can check this by plotting the data or checking the measures for kurtosis (how sharp the peak is) and skewdness (?) (if more than half the data is on one side of the peak).

share|improve this answer
2  
What levels of kurtosis and skewdness are acceptable to meet the assumption of normality? –  A Lion Jul 19 '10 at 19:38
5  
Most statistical methods assume normality, not of the data, but rather of an assumed random variable, e.g. the error term in a linear regression. Checking involves looking at the residuals, not the original data! –  Statprof Jul 19 '10 at 22:24

Other answers have covered what is normality and suggested normality test methods. Christian highlighted that in practice perfect normality barely exists.

I highlight that observed deviation from normality does not necessarily mean that methods assuming normality may not be used, and normality test may not be very useful.

  1. Deviation from normality may be caused by outliers that are due to errors in data collection. In many cases checking the data collection logs you can correct these figures and normality often improves.
  2. For large samples a normality test will be able to detect a negligible deviation from normality.
  3. Methods assuming normality may be robust to non-normality and give results of acceptable accuracy. The t-test is known to be robust in this sense, while the F test is not source(permalink). Concerning a specific method it's best to check the literature about robustness.
share|improve this answer
    
I think the reason why normality is a good assumption is because of its relative lack of use of the data - only the first two moments are used in estimation with the normal distribution. This makes diagnostic checking of a least squares model very easy - basically you just look for outliers which could influence the sufficient statistics. –  probabilityislogic Oct 6 '12 at 1:41

To add to the answers above: The "normality assumption" is that, in a model $Y=\mu+X\beta +\epsilon$, the residuak term $\epsilon$ is normally distributed. This assumption (as i ANOVA) often goes with some other: 2) The variance $\sigma^2$ of $\epsilon$ is constant, 3) independence of the observations.

Of this three assumptions, 2) and 3) are mostly vasly more important than 1)! So you should preoccupy yourself more with them. George Box said something in the line of ""To make a preliminary test on variances is rather like putting to sea in a row boat to find out whether conditions are sufficiently calm for an ocean liner to leave port!" - [Box, "Non-normality and tests on variances", 1953, Biometrika 40, pp. 318-335]"

This means that, unequal variances are of great concern, but actually testing for them is very difficult, because the tests are influenced by non-normality so small that it is of no importance for tests of means. Today, there are non-parametric tests for unequal variances that DEFINITELY should be used.

In short, preoccupy yourself FIRST about unequal variances, then about normality. When you have made yourself an opinion about them, you can think about normality!

Here is a lot of good advice: http://rfd.uoregon.edu/files/rfd/StatisticalResources/glm10_homog_var.txt

share|improve this answer
    
I am quite sure my interpretation is right. Box has also written at length about this in Box, Hunter & Hunter: Statistics for Experimenters which I have read thoroughly. But now I see that what I wrote about whas not what I meant, it should say ...then about normality! unequal variances are much more important than normality. Of course , independence is the mother of all assumptions. –  kjetil b halvorsen Oct 5 '12 at 22:06
1  
will edit, I saw the error in my answer while writing. –  kjetil b halvorsen Oct 5 '12 at 22:11

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.