# Limitations of Shapiro test

Are there any possible limitations to the Shapiro test or the shapiro.test() function? After removing outliers in a data set it seem that using the shapiro.test shows that the data is no longer normal, although the test showed that the model was normal before.

• Why removing outliers? Why in need of normal distribution? Jun 10, 2020 at 7:36
• @Stephan Kolassa rightly added a tag for outliers. Looking at some of the most upvoted threads under that tag will show that "removing outliers" is far from obvious or unproblematic. I'd say that the only uncontroversial reason for removing outliers is independent evidence that a value is wrong and there is no scope to correct it. Jun 10, 2020 at 7:41
• In addition to the Shapiro-Wilk test there is a Shapiro-Francia test. Please explain what software you are using, as until the whole world uses one statistical language or environment it does no harm to say which you are using and that could be informative to many readers. Jun 10, 2020 at 7:44

The Shapiro-Wilk test is not especially sensitive to outliers. There are normality tests that focus on outliers, by looking at a combination of skewness and kurtosis, but they are different.

Patrick Royston says about the Shapiro-Wilk test

Its power characteristics are well known and may be summarized by saying that it is strongest against short-tailed (platykurtic) and skew distributions and weakest against symmetric moderately long- tailed (leptokurtic) distributions.

An outlier-based test would be the opposite, strongest against leptokurtic distributions.

This also supports the idea that removing extreme observations from a sample that's truly from a Normal distribution might lead the Shapiro-Wilk test to reject -- correctly, since you no longer have a sample from a Normal distribution, but instead have a sample that has lighter tails than if it were from a Normal.

I have a longer rant about the Shapiro-Wilk test, which isn't necessarily relevant to the specific question here

• I love your quote: "The reason people test for fit to the Normal distribution is… well, to be honest, it’s typically because they read a statistics textbook written by a statistician who should have known better." Jun 10, 2020 at 8:23
• You were diplomatic enough not to say "or a statistics textbook written by a non-statistician who should have known better" but that fits many cases. I have often translated ad hoc as "fit for purpose" but that is not the pejorative flavour it usually has acquired. Jun 10, 2020 at 10:17

Certainly that can happen.

Assume your original data was normal. Really normal. Then remove "outliers". E.g., everything more than 3 standard deviations away from the mean. What are we left with? A truncated normal distribution. Which is, by definition, not a normal distribution any more.

So the Shapiro-Wilk test should in this situation tell you that the original data was normal, and the truncated data is not.

Here is a little R code to simulate this:

sample_size <- 100
n_experiments <- 1000
result <- matrix(NA,nrow=n_experiments,ncol=2,dimnames=list(NULL,c("Before","After")))

for ( ii in 1:n_experiments ) {
set.seed(ii)    # for replicability
foo <- rnorm(sample_size)
result[ii,"Before"] <- shapiro.test(foo)$$p.value>0.05 result[ii,"After"] <- shapiro.test(foo[abs(foo)<=3])$$p.value>0.05
}

table(data.frame(result))


Result:

       After
Before  FALSE TRUE
FALSE    32   26
TRUE      3  939


(I recommend skimming through the comments at this post to see how controversial "outlier" removal is.)

• Nice and simple! Jun 10, 2020 at 17:03