What is normality?

Question

In many different statistical methods there is an "assumption of normality". What is "normality" and how do I know if there is normality?

Answer 1

The assumption of normality is just the supposition that the underlying random variable of interest is distributed normally, or approximately so. Intuitively, normality may be understood as the result of the sum of a large number of independent random events.

More specifically, normal distributions are defined by the following function:

where $\mu$ and $\sigma^2$ are the mean and the variance, respectively, and which appears as follows:

This can be checked in multiple ways, that may be more or less suited to your problem by its features, such as the size of n. Basically, they all test for features expected if the distribution were normal (e.g. expected quantile distribution).

Answer 2

One note: The assumption of normality is often NOT about your variables, but about the error, which is estimated by the residuals. For example, in linear regression $Y = a + bx + e$; there is no assumption that $Y$ is normally distributed, only that $e$ is.

Answer 3

A related question can be found here about the normal assumption of the error (or more generally of the data if we do not have prior knowledge about the data).

Basically,

1. It is mathematically convenient to use normal distribution. (It's related to Least Squares fitting and easy to solve with pseudoinverse)
2. Due to Central Limit Theorem, we may assume that there are lots of underlying facts affecting the process and the sum of these individual effects will tend to behave like normal distribution. In practice, it seems to be so.

An important note from there is that, as Terence Tao states here, "Roughly speaking, this theorem asserts that if one takes a statistic that is a combination of many independent and randomly fluctuating components, with no one component having a decisive influence on the whole, then that statistic will be approximately distributed according to a law called the normal distribution".

To make this clear, let me write a Python code snippet

# -*- coding: utf-8 -*-
"""
Illustration of the central limit theorem

@author: İsmail Arı, http://ismailari.com
@date: 31.03.2011
"""

import scipy, scipy.stats
import numpy as np
import pylab

#===============================================================
# Uncomment one of the distributions below and observe the result
#===============================================================
x = scipy.linspace(0,10,11)
#y = scipy.stats.binom.pmf(x,10,0.2) # binom
#y = scipy.stats.expon.pdf(x,scale=4) # exp
#y = scipy.stats.gamma.pdf(x,2) # gamma
#y = np.ones(np.size(x)) # uniform
y = scipy.random.random(np.size(x)) # random

y = y / sum(y);

N = 3
ax = pylab.subplot(N+1,1,1)
pylab.plot(x,y)

# Plotting details 
ax.set_xticks([10])
ax.axis([0, 2**N * 10, 0, np.max(y)*1.1])
ax.set_yticks([round(np.max(y),2)])

#===============================================================
# Plots
#===============================================================
for i in np.arange(N)+1:
    y = np.convolve(y,y)
    y = y / sum(y);    

    x = np.linspace(2*np.min(x), 2*np.max(x), len(y))
    ax = pylab.subplot(N+1,1,i+1)
    pylab.plot(x,y)
    ax.axis([0, 2**N * 10, 0, np.max(y)*1.1])
    ax.set_xticks([2**i * 10])
    ax.set_yticks([round(np.max(y),3)])

pylab.show()

As can be seen from the figures, the resulting distribution (sum) tends towards a normal distribution regardless of the individual distribution types. So, if we do not have enough information about the underlying effects in the data, normality assumption is reasonable.

Answer 4

You can't know whether there normality and that's why you have to make an assumption that's there. You can only prove the absence of normality with statistic tests.

Even worse, when you work with real world data it's almost certain that there isn't true normality in your data.

That means that your statistical test is always a bit biased. The question is whether you can live with it's bias. To do that you have to understand your data and the kind of normality that your statistical tool assumes.

It's the reason why Frequentist tools are as subjective as Bayesian tools. You can't determine based on the data that it's normally distributed. You have to assume normality.

Answer 5

The assumption of normality assumes your data is normally distributed (the bell curve, or gaussian distribution). You can check this by plotting the data or checking the measures for kurtosis (how sharp the peak is) and skewdness (?) (if more than half the data is on one side of the peak).

Answer 6

Other answers have covered what is normality and suggested normality test methods. Christian highlighted that in practice perfect normality barely exists.

I highlight that observed deviation from normality does not necessarily mean that methods assuming normality may not be used, and normality test may not be very useful.

1. Deviation from normality may be caused by outliers that are due to errors in data collection. In many cases checking the data collection logs you can correct these figures and normality often improves.
2. For large samples a normality test will be able to detect a negligible deviation from normality.
3. Methods assuming normality may be robust to non-normality and give results of acceptable accuracy. The t-test is known to be robust in this sense, while the F test is not ^{source(permalink)}. Concerning a specific method it's best to check the literature about robustness.

Answer 7

To add to the answers above: The "normality assumption" is that, in a model $Y=\mu+X\beta +\epsilon$, the residuak term $\epsilon$ is normally distributed. This assumption (as i ANOVA) often goes with some other: 2) The variance $\sigma^2$ of $\epsilon$ is constant, 3) independence of the observations.

Of this three assumptions, 2) and 3) are mostly vasly more important than 1)! So you should preoccupy yourself more with them. George Box said something in the line of ""To make a preliminary test on variances is rather like putting to sea in a row boat to find out whether conditions are sufficiently calm for an ocean liner to leave port!" - [Box, "Non-normality and tests on variances", 1953, Biometrika 40, pp. 318-335]"

This means that, unequal variances are of great concern, but actually testing for them is very difficult, because the tests are influenced by non-normality so small that it is of no importance for tests of means. Today, there are non-parametric tests for unequal variances that DEFINITELY should be used.

In short, preoccupy yourself FIRST about unequal variances, then about normality. When you have made yourself an opinion about them, you can think about normality!

Here is a lot of good advice: http://rfd.uoregon.edu/files/rfd/StatisticalResources/glm10_homog_var.txt