# Are there any statistical tests for which exact critical values are unknown?

I know that, for Jarque-Bera Test of Normality, critical values are simulated if the sample size is relatively small.

Are there any other statistical tests for which critical values can only be found via Monte Carlo simulation?

Are there any statistical tests for which exact critical values are unknown?

It depends on what, exactly you mean by 'exact' and 'unknown'.

For example, the normal cdf and its inverse (from which you get critical values) cannot be written in terms of elementary functions. We can approximate it via various means (infinite series, continued fractions, various rational approximations). In that sense, "exact critical values" for the normal are only approximations.

for which critical values can only be found via Monte Carlo simulation?

"can only" implies that you couldn't do it any other way, even in principle, but there's no particular reason that many things we use simulation for couldn't be done in other ways. In some cases, numerical integration might be used, for example.

In many cases simulation is easy, and typically sufficient. That doesn't mean it's the only possibility - and even for cases where we might presently only have simulation available (there may be some), it doesn't imply that it will remain the only possibility.

If we relax that above notion of exactness and say "okay but having it accurate to some specified accuracy is sufficient" then you can essentially achieve that with simulated critical values also -- it may just take a lot of simulations to get high accuracy.

Certainly many tests have critical values and/or p-values obtained by simulation.

One example is the Lilliefors tests for the normal and exponential distribution.

This is also the case with many tests using permutation, for which the combinatorial explosion can make complete enumeration infeasible (though there are fast exact algorithms that have tamed many of these).

I know that, for Jarque-Bera Test of Normality, critical values are simulated if the sample size is relatively small.

You may be surprised to discover just how large the sample size is at which the bivariate normal approximation for the skewness and kurtosis becomes a reasonable approximation. Simulations along the lines of those in Bowman and Shenton's paper [1]* indicate that even at sample sizes of 200 or 300 the bivariate normal isn't a good approximation (500 is pretty reasonable to my eye). Bowman and Shenton suggest an approach that doesn't directly rely on simulation for smaller samples than those at which the asymptotic test is suitable (though one of their diagrams is strange; it looks as if something has been flipped over).

* (note carefully how much earlier this is than the paper by Jarque and Bera -- and this test was already quite old when Bowman and Shenton were explaining why the asymptotics didn't work even at moderate sample sizes and outlining what to do about it. Why econometricians choose to give priority to Jarque and Bera is puzzling. There are several better options.)

[1] Bowman, K.O., and Shenton, L.R. (1975).
"Omnibus contours for departures from normality based on √b1 and b2".
Biometrika. 62 (2): 243–250.

Are there any other statistical tests for which critical values can only be found via Monte Carlo simulation?

There must be, since all statistical tests is an "open", unfinished, set ... Mant tests is use are asymptotic, so the critical values used are only approximate. In such cases, to find exact critical levels would need some extra analysis, and in practice, maybe simulation.

In practice, in such cases, permutation tests is often a good alternative (when applicable.)

• yes there are many but one that comes to mind is the dickey fuiller unit root test. Under the null, the distribution of the statistic does not have a nice closed form so it's been simulated and tabulated. Jul 19, 2019 at 0:18
• @mlofton That case would at best be "has only" rather than "could only". Aug 27, 2023 at 5:07