34
$\begingroup$

In many different statistical methods there is an "assumption of normality". What is "normality" and how do I know if there is normality?

$\endgroup$
1

7 Answers 7

29
$\begingroup$

The assumption of normality is just the supposition that the underlying random variable of interest is distributed normally, or approximately so. Intuitively, normality may be understood as the result of the sum of a large number of independent random events.

More specifically, normal distributions are defined by the following function:

$$ f(x) =\frac{1}{\sqrt{2\pi\sigma^2}}e^{ -\frac{(x-\mu)^2}{2\sigma^2} },$$

where $\mu$ and $\sigma^2$ are the mean and the variance, respectively, and which appears as follows:

alt text

This can be checked in multiple ways, that may be more or less suited to your problem by its features, such as the size of n. Basically, they all test for features expected if the distribution were normal (e.g. expected quantile distribution).

$\endgroup$
0
38
$\begingroup$

One note: The assumption of normality is often NOT about your variables, but about the error, which is estimated by the residuals. For example, in linear regression $Y = a + bx + e$; there is no assumption that $Y$ is normally distributed, only that $e$ is.

$\endgroup$
4
  • 18
    $\begingroup$ +1. Finally someone has pointed out what perhaps is the most important aspect of this question: in most situations, "normality" is important in regard to residuals or to sampling distributions of statistics, not in regard to the distributions of the populations! $\endgroup$
    – whuber
    Oct 21, 2011 at 20:27
  • 5
    $\begingroup$ I would add that if $e$ is normally distributed, then Y is at least conditionally normal as well. I think its this that gets missed - people think that Y is marginally normal but its actually conditional normality that is needed. The simplest example of this is a one way ANOVA. $\endgroup$ Oct 6, 2012 at 1:34
  • $\begingroup$ Conditionally on what? $\endgroup$
    – bill_e
    Jan 28, 2014 at 5:46
  • 1
    $\begingroup$ @bill_e independent variables $\endgroup$
    – Glen_b
    Feb 4, 2017 at 1:47
10
$\begingroup$

A related question can be found here about the normal assumption of the error (or more generally of the data if we do not have prior knowledge about the data).

Basically,

  1. It is mathematically convenient to use normal distribution. (It's related to Least Squares fitting and easy to solve with pseudoinverse)
  2. Due to Central Limit Theorem, we may assume that there are lots of underlying facts affecting the process and the sum of these individual effects will tend to behave like normal distribution. In practice, it seems to be so.

An important note from there is that, as Terence Tao states here, "Roughly speaking, this theorem asserts that if one takes a statistic that is a combination of many independent and randomly fluctuating components, with no one component having a decisive influence on the whole, then that statistic will be approximately distributed according to a law called the normal distribution".

To make this clear, let me write a Python code snippet

# -*- coding: utf-8 -*-
"""
Illustration of the central limit theorem

@author: İsmail Arı, http://ismailari.com
@date: 31.03.2011
"""
 
import scipy, scipy.stats
import numpy as np
import pylab
 
#===============================================================
# Uncomment one of the distributions below and observe the result
#===============================================================
x = scipy.linspace(0,10,11)
#y = scipy.stats.binom.pmf(x,10,0.2) # binom
#y = scipy.stats.expon.pdf(x,scale=4) # exp
#y = scipy.stats.gamma.pdf(x,2) # gamma
#y = np.ones(np.size(x)) # uniform
y = scipy.random.random(np.size(x)) # random
 
y = y / sum(y);

N = 3
ax = pylab.subplot(N+1,1,1)
pylab.plot(x,y)

# Plotting details 
ax.set_xticks([10])
ax.axis([0, 2**N * 10, 0, np.max(y)*1.1])
ax.set_yticks([round(np.max(y),2)])
 
#===============================================================
# Plots
#===============================================================
for i in np.arange(N)+1:
    y = np.convolve(y,y)
    y = y / sum(y);    
 
    x = np.linspace(2*np.min(x), 2*np.max(x), len(y))
    ax = pylab.subplot(N+1,1,i+1)
    pylab.plot(x,y)
    ax.axis([0, 2**N * 10, 0, np.max(y)*1.1])
    ax.set_xticks([2**i * 10])
    ax.set_yticks([round(np.max(y),3)])
 
pylab.show()

Random distribution

Exponential distribution

Uniform distribution

As can be seen from the figures, the resulting distribution (sum) tends towards a normal distribution regardless of the individual distribution types. So, if we do not have enough information about the underlying effects in the data, normality assumption is reasonable.

$\endgroup$
1
  • 1
    $\begingroup$ The CLT doesn't allow us to assume there are lots of individual effects in any given process - if we are given that there are lots of not-too-dependent individual factors contributing to a measurement (none of which have too much of the total variation), we may be justified in assuming normality by invoking the CLT. The assumption of many contributions precedes the application of the CLT, it's not in any sense a result of the CLT. Otherwise everything would be normal, when in fact that's only sometimes roughly true. $\endgroup$
    – Glen_b
    May 21, 2014 at 6:01
6
$\begingroup$

You can't know whether there normality and that's why you have to make an assumption that's there. You can only prove the absence of normality with statistic tests.

Even worse, when you work with real world data it's almost certain that there isn't true normality in your data.

That means that your statistical test is always a bit biased. The question is whether you can live with it's bias. To do that you have to understand your data and the kind of normality that your statistical tool assumes.

It's the reason why Frequentist tools are as subjective as Bayesian tools. You can't determine based on the data that it's normally distributed. You have to assume normality.

$\endgroup$
5
  • 5
    $\begingroup$ You cannot prove anything using statistics. A proof is meant to be exact. Statistics is about probabilities. Even a p=0.99 result of a Chi squared does not "prove" that the underlying distribution is not normal. It's just damn unlikely that it's normal. $\endgroup$
    – xmjx
    Jun 16, 2011 at 7:53
  • $\begingroup$ @xmjx: You can't even say that a given distribution is probably normal distributed. If you have a distribution with where 99.99% of your values are 1 but 0.01% of your values are 1000000 a statistic test that samples 100 values has a good chance to tell you wrongly that your distribution is normally distributed. $\endgroup$
    – Christian
    Oct 21, 2011 at 19:44
  • 2
    $\begingroup$ I'm not much of a statistical expert, so this may seem like a silly question... doesn't "true normality" exist in the underlying process that generates the variable rather than the data? It may seem like a silly distinction, but perhaps it could save some soul-searching. If the gathered data is not exactly normal, but the underlying random process works in a basically normal way, is that a situation where you could decide to "live with the bias"? $\endgroup$
    – Jonathan
    Oct 21, 2011 at 21:28
  • $\begingroup$ @Christian - your comment that "...100 values has a good chance..." isn't borne out at all by my hacking: x=c(rep(1,99),rep(1000000,1)); ks.test(x,pnorm) > The assumption of normality is still "rejected" by the K-S Test. $\endgroup$
    – rolando2
    Oct 5, 2012 at 22:27
  • 1
    $\begingroup$ I like this answer (+1) but it is a bit pessimistic about what can be done with the assumption of normality. It is usually a good starting point for any modelling, and you can generalise to a very wide class of distributions by taking either mixtures or functions of normally distributed random variables. $\endgroup$ Oct 6, 2012 at 1:46
3
$\begingroup$

The assumption of normality assumes your data is normally distributed (the bell curve, or gaussian distribution). You can check this by plotting the data or checking the measures for kurtosis (how sharp the peak is) and skewdness (?) (if more than half the data is on one side of the peak).

$\endgroup$
2
  • 2
    $\begingroup$ What levels of kurtosis and skewdness are acceptable to meet the assumption of normality? $\endgroup$
    – A Lion
    Jul 19, 2010 at 19:38
  • 5
    $\begingroup$ Most statistical methods assume normality, not of the data, but rather of an assumed random variable, e.g. the error term in a linear regression. Checking involves looking at the residuals, not the original data! $\endgroup$
    – Statprof
    Jul 19, 2010 at 22:24
3
$\begingroup$

Other answers have covered what is normality and suggested normality test methods. Christian highlighted that in practice perfect normality barely exists.

I highlight that observed deviation from normality does not necessarily mean that methods assuming normality may not be used, and normality test may not be very useful.

  1. Deviation from normality may be caused by outliers that are due to errors in data collection. In many cases checking the data collection logs you can correct these figures and normality often improves.
  2. For large samples a normality test will be able to detect a negligible deviation from normality.
  3. Methods assuming normality may be robust to non-normality and give results of acceptable accuracy. The t-test is known to be robust in this sense, while the F test is not source. Concerning a specific method it's best to check the literature about robustness.
$\endgroup$
1
  • 1
    $\begingroup$ I think the reason why normality is a good assumption is because of its relative lack of use of the data - only the first two moments are used in estimation with the normal distribution. This makes diagnostic checking of a least squares model very easy - basically you just look for outliers which could influence the sufficient statistics. $\endgroup$ Oct 6, 2012 at 1:41
3
$\begingroup$

To add to the answers above: The "normality assumption" is that, in a model $Y=\mu+X\beta +\epsilon$, the residual term $\epsilon$ is normally distributed. This assumption (as in ANOVA) often goes with some other:
2. The variance $\sigma^2$ of $\epsilon$ is constant,
3. independence of the observations.

Of this three assumptions, 2) and 3) are mostly vastly more important than 1)! So you should preoccupy yourself more with them. George Box said something in the line of "To make a preliminary test on variances is rather like putting to sea in a row boat to find out whether conditions are sufficiently calm for an ocean liner to leave port!" - [Box, "Non-normality and tests on variances", 1953, Biometrika 40, pp. 318-335]"

This means that, unequal variances are of great concern, but actually testing for them is very difficult, because the tests are influenced by non-normality so small that it is of no importance for tests of means. Today, there are non-parametric tests for unequal variances that DEFINITELY should be used.

In short, preoccupy yourself FIRST about unequal variances, then about normality. When you have made yourself an opinion about them, you can think about normality!

A lot of good advice can be found here.

$\endgroup$
1
  • $\begingroup$ I am quite sure my interpretation is right. Box has also written at length about this in Box, Hunter & Hunter: Statistics for Experimenters which I have read thoroughly. But now I see that what I wrote about whas not what I meant, it should say ...then about normality! unequal variances are much more important than normality. Of course , independence is the mother of all assumptions. $\endgroup$ Oct 5, 2012 at 22:06

Not the answer you're looking for? Browse other questions tagged or ask your own question.