sample size of Anderson-Darling test

Question

Why must the sample size of the Anderson-Darling test for normality be greater than 7? The function ad.test in some packages in R enforce a sample size $n>7$.

I want to test a data set of 5 points in a normality test, although the meaning might be nothing.

Answer 1

A direct answer comes from Anderson and Darling in their paper on the subject:

Significance points for $W_n^2$ [the test statistic] are not available for small sample sizes.

A sample size of five is just enough to create a five-number summary of the data. With such a small sample size, it can be difficult to describe the shape of the data with any meaningful certainty. With such a small sample size, it would be very probable to have false negatives of the test, as well as false positives.

This can be easily evaluated in R by making tables of random inverse normal data sets of five points each and running the Anderson-Darling test. In my tests, I found the false negatives to be around three to ten percent. False positives of a $\chi^2$ distribution with four degrees of freedom or a $\Gamma$ distribution with $\alpha=4$ and $\beta=5$ were above 90%. (To be honest, the data is not much better when using a sample size of 7, but there is a larger difference between the distributions.)

The basic idea of most tests is to compare a sample set of data with the desired distribution and see how well the data fits the theoretical. A lot of tests are weighted to look at the tails of the distribution. With only five of seven data points, the assumption is that only one point defines each tail, which makes it very easy to fit almost any data to a Gaussian distribution.

NIST provides this sample R code with results from four distributions. The same tests can be run again with a sample size of 8 and a seed of (403) and all four distributions return as "normal."

Answer 2

To summarize the discussion in the comments, I am not certain where the minimum value in the nortest package comes from. The references in that package are Testing for Normality (Thode 2002) and Goodness-of-Fit-Techniques (D'Agostino & Stephens 1986). In the former, the appropriate section (5.1.4) was available on Google Books, and I saw no mention of a minimum sample size. In the latter, I've re-skimmed section 4, which discusses EDF-based tests, and also saw no mention. As a matter of fact, there is at least one mention of $n = 5$ (p. 160) regarding the Anderson-Darling.

The ad.test in the ADGofTest package is based on Evaluating the Anderson-Darling Distribution (Marsaglia & Marsaglia 2004) which clearly allows its use for any $n$. Nor does the documentation in that test give a minimum number.

In your case, you may feel most comfortable using the ad.test of the ADGofTest package.

Update

I have found a source for a sample size on AD. Lewis (1961, p. 1120) states "…for practical purposes, the asymptotic distribution can be used for $n > 8$."

• Lewis, Peter A. W. Distribution of the Anderson-Darling Statistic. Ann. Math. Statist. 32 (1961), no. 4, 1118--1124. doi:10.1214/aoms/1177704850. http://projecteuclid.org/euclid.aoms/1177704850.

Answer 3

For samples from a location–scale family, the distribution of the Anderson–Darling statistic calculated with location & scale parameters estimated from the sample depends only on the family & the sample size. So a single simulation, as suggested by @Glen_b, from the standard normal distribution will suffice to give you exact critical values or p-values for a given sample size to any desired precision.

library(ADGofTest)
no.sims <- 9999
ad.stat <- numeric(no.sims)
for (i in 1:no.sims){
  #simulate sample of 5 standard normal variates
  x <- rnorm(5)
  #calculate test statistic, estimating mean & standard deviation from sample
  ad.test(x, distr.fun=pnorm, mean=mean(x), sd=sd(x))$stat -> ad.stat[i]
}
# calculate critical values
quantile(ad.stat, prob=c(0.9, 0.95, 0.99))

So for a sample size of five, the 90%, 95%, & 99% critical values of the test statistic are around 0.52, 0.60. & 0.80. As @Tavrock points out, the power to detect even quite large departures from normality will be low.

A comparison of this simulated distribution with the approximate one used in ad.test from nortest suggests the package writers were erring on the side of caution when they decided to disallow sample sizes less than seven: