Does there always exist for n small, a non-chi-squared test-statistic for the likelihood-ratio (neyman-pearson, karlin-rubin), score, and wald-tests?

Question

An additional reason that the chi-squared distribution is widely used is that it turns up as the large sample distribution of generalized likelihood ratio tests (LRT).[6] LRTs have several desirable properties; in particular, simple LRTs commonly provide the highest power to reject the null hypothesis (Neyman–Pearson lemma) and this leads also to optimality properties of generalised LRTs. However, the normal and chi-squared approximations are only valid asymptotically. For this reason, it is preferable to use the t distribution rather than the normal approximation or the chi-squared approximation for a small sample size. Similarly, in analyses of contingency tables, the chi-squared approximation will be poor for a small sample size, and it is preferable to use Fisher's exact test. Ramsey shows that the exact binomial test is always more powerful than the normal approximation.[7] :https://en.wikipedia.org/wiki/Chi-squared_distribution

By the central limit theorem, because the chi-squared distribution is the sum of $k$ independent random variables with finite mean and variance, it converges to a normal distribution for large $k$. For many practical purposes, for $k>50$ the distribution is sufficiently close to a normal distribution for the difference to be ignored. ${ }^{[13]}$ Specifically, if $X \sim \chi^{2}(k)$, then as $k$ tends to infinity, the distribution of $(X-k) / \sqrt{2 k}$ tends to a standard normal distribution. However, convergence is slow as the skewness is $\sqrt{8 / k}$ and the excess kurtosis is $12 / k$

Does there always exist a non-chi-squared test-statistic for the likelihood-ratio (neyman-pearson, karlin-rubin), score, and wald-tests?

In the nested-mixed models/gee, is there a better statistic than the chi-square from asymptotic theory?

Is there a method/strategy for identifying the (non-chi-squared) test-statistic in these tests? I want to avoid asymptotic distributions as I never have infinite data.

Does the uniformly most powerful property of the neyman-pearson and wilks-rubin property depend on avoiding the asymptotic chi-square distribution of the test-statistic in small sample sizes and use the correct distribution for the test-statistic or are these tests even more powerful when one chooses the correct distribution in small-samples?

Answer 1

While $n$ is never infinite, as a practical matter it is in many cases large enough to give usable results.

It seems kind of odd to worry overly much about the exactness and power of the small sample test when the distributional model assumption itself will (almost always) be an approximation; perhaps it would make more sense to worry about its robustness to plausible deviations from that model. That is, it's fine to be worried about whether the impact of sample size is large or small when the model is correct, but the larger problem may be that the model is not correct.

Algebraic manipulations

Nevertheless, with likelihood ratio statistics (or indeed the score statistic and so forth) we can sometimes obtain exact small sample distributions; this would be considered on a case-by-case basis but the more complicated the model the harder it typically becomes.

The first issue with the asymptotic distribution in small samples is accuracy of significance level. Naturally if you conduct tests based off the same statistic at different significance levels, their power curves differ. Control of significance levels may be important.

Whether you use $\Lambda$ or $-2\log \Lambda$ is not material (they're equivalent statistics), what matters is where you place your critical value for each; obtaining a critical value from the asymptotic chi-squared result is not going to be exact for finite samples (though it may be fine for some purpose).

We might expect that the Wald and score tests may tend to have less power in small samples (once we get the attained significance levels to be the same), though sometimes there will be little or no difference.

Simulation

You may be able to use simulation from the null model (plus all the relevant assumptions) to identify suitable rejection regions in many cases; this can be made as accurate as you require by simulating more.

Nonparametric tests based on the likelihood ratio (etc) statistic

There's another possibility with finite samples, which is to use such statistics as the basis of a resampling test. In many simple cases you may be able to perform a permutation test using such a statistic (yielding an 'exact' test - irrespective of the distribution - with good power when the model is correct); alternatively there's the possibility of bootstrapping in large samples. Various common techniques can improve the approximation of the significance level for the bootstrap in smaller samples.