122

I think that tests for normality can be useful as companions to graphical examinations. They have to be used in the right way, though. In my opinion, this means that many popular tests, such as the Shapiro-Wilk, Anderson-Darling and Jarque-Bera tests never should be used. Before I explain my standpoint, let me make a few remarks: In an interesting recent ...


103

@DikranMarsupial is exactly right, of course, but it occurred to me that it might be nice to illustrate his point, especially since this concern seems to come up frequently. Specifically, the residuals of a regression model should be normally distributed for the p-values to be correct. However, even if the residuals are normally distributed, that doesn't ...


74

John Tukey advocated his "three point method" for finding re-expressions of variables to linearize relationships. I will illustrate with an exercise from his book, Exploratory Data Analysis. These are mercury vapor pressure data from an experiment in which temperature was varied and vapor pressure was measured. pressure <- c(0.0004, 0.0013, 0.006, 0.03,...


59

Let $X_1, ..., X_n \sim N(\mu, \sigma^2)$. As shown in this thread, the standard deviation of the sample standard deviation, $$ s = \sqrt{ \frac{1}{n-1} \sum_{i=1}^{n} (X_i - \overline{X}) }, $$ is $$ {\rm SD}(s) = \sqrt{ E \left( [E(s)- s]^2 \right) } = \sigma \sqrt{ 1 - \frac{2}{n-1} \cdot \left( \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } \...


56

The ordinary least squares estimate is still a reasonable estimator in the face of non-normal errors. In particular, the Gauss-Markov Theorem states that the ordinary least squares estimate is the best linear unbiased estimator (BLUE) of the regression coefficients ('Best' meaning optimal in terms of minimizing mean squared error)as long as the errors (1) ...


45

Note that the Shapiro-Wilk is a powerful test of normality. The best approach is really to have a good idea of how sensitive any procedure you want to use is to various kinds of non-normality (how badly non-normal does it have to be in that way for it to affect your inference more than you can accept). An informal approach for looking at the plots would ...


35

I presume you mean the F-test for the ratio of variances when testing a pair of sample variances for equality (because that's the simplest one that's quite sensitive to normality; F-test for ANOVA is less sensitive) If your samples are drawn from normal distributions, the sample variance has a scaled chi square distribution Imagine that instead of data ...


34

We usually know it's impossible for a variable to be exactly normally distributed... The normal distribution has infinitely long tails extending out in either direction - it is unlikely for data to lie far out in these extremes, but for a true normal distribution it has to be physically possible. For ages, a normally distributed model will predict there is ...


33

What happens if the residuals are not homoscedastic? If the residuals show an increasing or decreasing pattern in Residuals vs. Fitted plot. If the error term is not homoscedastic (we use the residuals as a proxy for the unobservable error term), the OLS estimator is still consistent and unbiased but is no longer the most efficient in the class of linear ...


33

Let me start by denying the premise. Robert Geary probably didn't overstate the case when he said (in 1947) "...normality is a myth; there never was, and never will be, a normal distribution." -- the normal distribution is a model*, an approximation that is sometimes more-or-less useful. $\:$*(about which, see George Box, though I prefer the version on my ...


30

I think that pre-testing for normality (which includes informal assessments using graphics) misses the point. Users of this approach assume that the normality assessment has in effect a power near 1.0. Nonparametric tests such as the Wilcoxon, Spearman, and Kruskal-Wallis have efficiency of 0.95 if normality holds. In view of 2. one can pre-specify the use ...


27

First off, I would get yourself a copy of this classic and approachable article and read it: Anscombe FJ. (1973) Graphs in statistical analysis The American Statistician. 27:17–21. On to your questions: Answer 1: Neither the dependent nor independent variable needs to be normally distributed. In fact they can have all kinds of loopy distributions. The ...


26

In Econometrics, we would say that non-normality violates the conditions of the Classical Normal Linear Regression Model, while heteroskedasticity violates both the assumptions of the CNLR and of the Classical Linear Regression Model. But those that say "...violates OLS" are also justified: the name Ordinary Least-Squares comes from Gauss directly and ...


25

I am not aware of an "official" definition and even if there it is, you shouldn't trust it as you will see it being used inconsistently in practice. This being said, scaling in statistics usually means a linear transformation of the form $f(x) = ax+b$. Normalizing can either mean applying a transformation so that you transformed data is roughly normally ...


25

Boxplots Here is a relevant section from Hoaglin, Mosteller and Tukey (2000): Understanding Robust and Exploratory Data Analysis. Wiley. Chapter 3, "Boxplots and Batch Comparison", written by John D. Emerson and Judith Strenio (from page 62): [...] Our definition of outliers as data values that are smaller than $F_{L}-\frac{3}{2}d_{F}$ or larger than $...


23

You can't really even compare the two since the Kolmogorov-Smirnov is for a completely specified distribution (so if you're testing normality, you must specify the mean and variance; they can't be estimated from the data*), while the Shapiro-Wilk is for normality, with unspecified mean and variance. * you also can't standardize by using estimated parameters ...


23

Without contradicting any of the excellent answers here, I have one rule of thumb which is often (but not always) decisive. (A passing comment in the answer by @Dante seems pertinent too.) It sometimes seems too obvious to state, but here you are. I am happy to call a distribution non-normal if I think I can offer a different description that is clearly ...


22

If we had time in the class where we first introduce regression models to discuss bootstrapping and the other techniques that you mentioned (including all their assumptions, pitfalls, etc.), then I would agree with you that it is not necessary to talk about normality and homoscedasticity assumptions. But in truth, when regression is first introduced we do ...


20

There is a famous saying by Gabriel Lippmann (physicist, Nobel laureate), as told by Poincaré: [The normal distribution] cannot be obtained by rigorous deductions. Several of its putative proofs are awful [...]. Nonetheless, everyone believes it, as M. Lippmann told me one day, because experimenters imagine it to be a mathematical theorem, while ...


19

Some books state a sample size of size 30 or higher is necessary for the central limit theorem to give a good approximation for $\bar{X}$. This common rule of thumb is pretty much completely useless. There are non-normal distributions for which n=2 will do okay and non-normal distributions for which much larger $n$ is insufficient - so without an explicit ...


19

1) rarely do people only want to estimate. Usually inference - CIs, PIs, tests - is the aim, or at least part of it (even if sometimes it's done relatively informally) 2) Things like the Gauss Markov theorem isn't necessarily much help -- if the distribution is sufficiently far from normal, a linear estimator is not much use. There's no point in getting the ...


19

Sure it can: To see this, all you need to do is ask the equivalent question: if I started with a set of values that are normally distributed, could I add more values that stuff this up? Obviously the answer to this question is yes, and since the former set is the subset of the whole, your answer follows.


17

First a general comment: Note that the Anderson-Darling test is for completely specified distributions, while the Shapiro-Wilk is for normals with any mean and variance. However, as noted in D'Agostino & Stephens$^{[1]}$ the Anderson-Darling adapts in a very convenient way to the estimation case, akin to (but converges faster and is modified in a way ...


17

Categorical data are not from a normal distribution. The normal distribution only makes sense if you're dealing with at least interval data, and the normal distribution is continuous and on the whole real line. If any of those aren't true you don't need to examine the data distribution to conclude that it's not consistent with normality. [Note that if it's ...


16

Before asking whether a test or any sort of rough check for normality is "useful" you have to answer the question behind the question: "Why are you asking?" For example, if you only want to put a confidence limit around the mean of a set of data, departures from normality may or not be important, depending on how much data you have and how big the ...


16

Questions about robustness are very hard to answer well - because the assumptions may be violated in so many ways, and in each way to different degrees. Simulation work can only sample a very small portion of the possible violations. Given the state of computing, I think it is often worth the time to run both a parametric and a non-parametric test, if both ...


15

Scaling is a personal choice about making the numbers feel right, e.g. between zero and one, or one and a hundred. For example converting data given in millimeters to meters because it's more convenient, or imperial to metric. While normalisation is about scaling to an external 'standard' - the local norm - such as removing the mean value and dividing by ...


15

In practice you simply don't know (but they probably aren't). Not that non-normal residuals are necessarily a problem; it depends on how non-normal and how big your sample size is and how much you care about the impact on your inference. You can see if the residuals are reasonably close to normal via a Q-Q plot. A Q-Q plot isn't hard to generate in Excel. ...


15

There are several misunderstandings in you post (some of which are common and you may have been told the wrong thing because the person telling you was just passing on the misinformation). First is that bootstrap is not the savior of the small sample size. Bootstrap actually fairs quite poorly for small sample sizes, even when the population is normal. ...


Only top voted, non community-wiki answers of a minimum length are eligible