6
$\begingroup$

I've found critical values for the Anderson Darling test for a Normal Distribution at 1%, 2.5%, 5%, 10% and 15% significance levels from various sources, including wikipedia:

http://en.wikipedia.org/wiki/Anderson%E2%80%93Darling_test

I'd really like a critical value for a 0.1% significance level (in "case 4" - neither mean nor variance known). I couldn't find it by searching the web, and I am not sure how I should calculate it. Can anyone help?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
5
$\begingroup$

Using Marsaglia & Marsaglia's code, and a bisection search, one can find that the 0.001 critical value is around 5.9694. This would be for 'Case 1' in the wikipedia article quoted. I am not sure how to convert to 'Case 4'.

Improve this answer
$\endgroup$
5
  • $\begingroup$ This appears to be an asymptotic value. A good simulation for your $n$ very well might be more accurate! $\endgroup$
    – whuber ♦
    Commented Oct 11, 2011 at 21:47
  • $\begingroup$ @whuber, indeed it is. Rahman et. al. give simulation values for $n=10,20,\ldots,100$ in table 2 of this paper: math.usm.my/bulletin/pdf/v29n1/v29n1p2.pdf , although, again, it does not address 'case 4' testing, just 'case 1'. $\endgroup$
    – shabbychef
    Commented Oct 11, 2011 at 22:53
  • $\begingroup$ Those simulation values don't even have two significant figures of accuracy. For instance, the line for $n=10$ in Table 2 should read 1.246, 1.624, 1.942, 2.512, 3.108, 3.924, 4.556, 6.044 ($10^7$ iterations). (The last digit in the higher values is still uncertain.) This took just one minute of computing time; the simulation takes only five times longer for $n=1000$. $\endgroup$
    – whuber ♦
    Commented Oct 11, 2011 at 23:13
  • $\begingroup$ It is troubling that e.g. the columns of that table do not appear to be monotonic (I would guess they are), which confirms the lack of accuracy, sadly. $\endgroup$
    – shabbychef
    Commented Oct 12, 2011 at 17:22
  • $\begingroup$ There is no variation (down columns) in that table that cannot be explained by uncertainty due to the small simulations that were performed (merely $10^5$ iterations in each). (The authors' attempt to find a pattern was doomed from the outset, but apparently they did not understand how much random error is present in this table.) In order to see the variation that might be there, I guess you would need around $10^8$ to $10^9$ iterations in each simulation. $\endgroup$
    – whuber ♦
    Commented Oct 12, 2011 at 17:30
5
$\begingroup$

You can use simulation (this is not a new idea, it is how Gosset/Student derived the original t table (but we have faster tools than he did)).

Generate a psuedo random sample from a normal distribution (or at least as close as the computer can come) of the sample size of interest and compute the Anderson Darling Statistic for that sample. Now repeate this process a few million times (or maybe more than a few depending on how precise you want to be). The 0.1% critical value will be the 0.1% or 99.9% percentile.

However, I have a hard time imagining a useful question that would be answered by an Anderson-Darling test at 0.1% significance. What is the question that you are trying to answer?

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ I see your point - this is an odd thing to do. I'm testing output from a simulation under a very large number of possible configurations, so I expect failures at 1% significance level. I thought of combining output from every configuration to get a single AD* number. I didn't do this because it wouldn't give me any hint as to which individual configuration was buggy, and because buggy configurations have (so far) failed this test spectacularly. I can see that by using a 0.1% significance level, I will admit some bugs. Perhaps I need to look the distribution of the individual AD* results? $\endgroup$
    – Matt
    Commented May 27, 2011 at 15:59
  • $\begingroup$ Re Greg's comment (+1): I wonder how much extreme critical values like .1% are impacted by different means and/or variances of the underlying normal distribution. After all, Matt mentions that he is in "case 4". $\endgroup$
    – Stephan Kolassa
    Commented May 27, 2011 at 19:55
  • $\begingroup$ OK, testing simulation results is one of the few cases that testing for matches of an exact distribution makes sense. But rather than testing at such a low alpa, you might consider looking at all your test statitics in some form of plot to look for unusual points (clumps could mean something different as well). $\endgroup$
    – Greg Snow
    Commented May 28, 2011 at 15:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.