6
$\begingroup$

Why must the sample size of the Anderson-Darling test for normality be greater than 7? The function ad.test in some packages in R enforce a sample size $n>7$.

I want to test a data set of 5 points in a normality test, although the meaning might be nothing.

$\endgroup$
10
  • 3
    $\begingroup$ Why do you want to test normality at n=5, and why must it be Anderson-Darling? $\,$ "Why was something implemented a particular way"-type questions pretty much require us to speculate on the thinking of someone (though occasionally a reason might be clear). It might be that way in R simply because the asymptotic approximation with estimated parameters works well down about that far. Or it might be for some other reason. (You can always simulate it at n=5.) $\endgroup$
    – Glen_b
    Commented Jan 28, 2015 at 2:22
  • $\begingroup$ It's not in base R -- which package and which function are you using? $\endgroup$
    – Glen_b
    Commented Jan 28, 2015 at 2:29
  • $\begingroup$ @Glen_b ad.test is in nortest. $\endgroup$
    – Avraham
    Commented Jan 28, 2015 at 2:45
  • $\begingroup$ @Avraham I'm curious - how do you know the OP is not looking at (for example) the ad.test in ADGofTest? $\endgroup$
    – Glen_b
    Commented Jan 28, 2015 at 2:47
  • 1
    $\begingroup$ @Glen_b Because nortest (pdf) is the one where the sample size of 7 is listed. The ad.test in ADGofTest (pdf) does not mention a minimum sample size. $\endgroup$
    – Avraham
    Commented Jan 28, 2015 at 2:53

3 Answers 3

2
$\begingroup$

A direct answer comes from Anderson and Darling in their paper on the subject:

Significance points for $W_n^2$ [the test statistic] are not available for small sample sizes.

A sample size of five is just enough to create a five-number summary of the data. With such a small sample size, it can be difficult to describe the shape of the data with any meaningful certainty. With such a small sample size, it would be very probable to have false negatives of the test, as well as false positives.

This can be easily evaluated in R by making tables of random inverse normal data sets of five points each and running the Anderson-Darling test. In my tests, I found the false negatives to be around three to ten percent. False positives of a $\chi^2$ distribution with four degrees of freedom or a $\Gamma$ distribution with $\alpha=4$ and $\beta=5$ were above 90%. (To be honest, the data is not much better when using a sample size of 7, but there is a larger difference between the distributions.)

The basic idea of most tests is to compare a sample set of data with the desired distribution and see how well the data fits the theoretical. A lot of tests are weighted to look at the tails of the distribution. With only five of seven data points, the assumption is that only one point defines each tail, which makes it very easy to fit almost any data to a Gaussian distribution.

NIST provides this sample R code with results from four distributions. The same tests can be run again with a sample size of 8 and a seed of (403) and all four distributions return as "normal."

$\endgroup$
2
  • $\begingroup$ The Anderson-Darling paper gives only asymptotic critical values: I think "not available for small sample sizes" means just that they hadn't tried to calculate exact ones, not that there's a problem in principle with doing so. $\endgroup$ Commented Jan 15, 2017 at 23:46
  • $\begingroup$ @Scortchi, regardless of if there was a problem in principle or a technical issue with calculating values for a small sample size, the net result was the same: it had not been done. $\endgroup$
    – Tavrock
    Commented Jan 16, 2017 at 15:37
1
$\begingroup$

To summarize the discussion in the comments, I am not certain where the minimum value in the nortest package comes from. The references in that package are Testing for Normality (Thode 2002) and Goodness-of-Fit-Techniques (D'Agostino & Stephens 1986). In the former, the appropriate section (5.1.4) was available on Google Books, and I saw no mention of a minimum sample size. In the latter, I've re-skimmed section 4, which discusses EDF-based tests, and also saw no mention. As a matter of fact, there is at least one mention of $n = 5$ (p. 160) regarding the Anderson-Darling.

The ad.test in the ADGofTest package is based on Evaluating the Anderson-Darling Distribution (Marsaglia & Marsaglia 2004) which clearly allows its use for any $n$. Nor does the documentation in that test give a minimum number.

In your case, you may feel most comfortable using the ad.test of the ADGofTest package.

Update

I have found a source for a sample size on AD. Lewis (1961, p. 1120) states "…for practical purposes, the asymptotic distribution can be used for $n > 8$."

$\endgroup$
1
  • 1
    $\begingroup$ Note that the ad.test of the ADGofTest package is for when the distribution's fully specified under the null, whereas the ad.test of the nortest package is for when mean & standard deviation are estimated from the data. So they're not equivalent. $\endgroup$ Commented Jan 16, 2017 at 0:01
1
+50
$\begingroup$

For samples from a location–scale family, the distribution of the Anderson–Darling statistic calculated with location & scale parameters estimated from the sample depends only on the family & the sample size. So a single simulation, as suggested by @Glen_b, from the standard normal distribution will suffice to give you exact critical values or p-values for a given sample size to any desired precision.

library(ADGofTest)
no.sims <- 9999
ad.stat <- numeric(no.sims)
for (i in 1:no.sims){
  #simulate sample of 5 standard normal variates
  x <- rnorm(5)
  #calculate test statistic, estimating mean & standard deviation from sample
  ad.test(x, distr.fun=pnorm, mean=mean(x), sd=sd(x))$stat -> ad.stat[i]
}
# calculate critical values
quantile(ad.stat, prob=c(0.9, 0.95, 0.99))

So for a sample size of five, the 90%, 95%, & 99% critical values of the test statistic are around 0.52, 0.60. & 0.80. As @Tavrock points out, the power to detect even quite large departures from normality will be low.

A comparison of this simulated distribution with the approximate one used in ad.test from nortest suggests the package writers were erring on the side of caution when they decided to disallow sample sizes less than seven:

distribution functions, simulated & approximate

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.