# Is it correct to use a summary statistic with known distribution under the null hypothesis as a test statistic?

How can a test statistic be found?

Suppose that we have a sample, and we want to have a simple hypothesis. We know that under $H_{0}$ one of the summaries (which is a random variable) of this sample has a known distribution. Is it correct to use this summary as a test statistic?

• Could you try to make your question more precise? Providing example illustrating your question would probably help.
– Tim
Commented Mar 22, 2016 at 9:37
• The short answer is yes (but I definitely agree with @Tim that more detail is needed). For your test to have nontrivial power, the summary would also need to have another distribution under the alternative hypothesis. Commented Mar 22, 2016 at 9:45
• I don't have an example that comes to my mind right now, but let's say we have a sample of population, and that the mean is a summary for this population, let's also assume, for the sake of explanation, that the mean is a random variable $S$ (which is not). Under $H_{0}$ the distribution of $S$ is $A$. Can I use $S$ directly as a test statistic? I mean by that computing the observed value of $S$ and then comparing it to the quantile of the distribution of $A$ in order to determine if I should accept or reject $H_{0}$. Commented Mar 22, 2016 at 9:59
• @ToneyShields basically "yes", if you know the null distribution, you can compare some value to it's null to run a test. But it is not totally clear from your question what do you mean -- an by this you could get an answer that answers different question that you're asking.
– Tim
Commented Mar 22, 2016 at 10:44
• Okay, can you please reffer me to some articles that propose their own statistic tests, I want to see what do they do in order to come up with the test statistic, how do they know it's distribution. Commented Mar 22, 2016 at 10:59

We know that under H0 one of the summaries (which is a random variable) of this sample has a known distribution. Is it correct to use this summary as a test statistic?

It kind of depends partly on what you mean by "correct".

You can use a statistic whose distribution is known under the null to perform a test of known size; in that sense it will be "correct".

However, unless that summary statistic relates to the hypothesis you're testing, it may make for a useless test (that is, a test can be 'correct' in the sense of having the desired significance level, but be of no practical use).

So for example, under an assumption of normality and given a null hypothesis about the mean, I know the distribution of the sample variance, but it's useless to use it for my test about the mean. You need a statistic that will have a different distribution under the alternative, and specifically, have a rejection region that the test statistic will fall into more often under the alternative than under the null.

Even when the statistic does "relate" to the population quantity (or quantities) in the null, you would want to consider the properties of the resulting test (power, test bias and so on) to assess its suitability, especially where alternative possibilities for test statistics can be identified.

[Note that you don't necessarily need to know the distribution of your test statistic under the null to use it as a test statistic; for example permutation tests or other resampling tests (e.g. bootstrap tests) can often be performed.]

• Thanks, I understand better now. I have another question if you don't mind: Suppose that we have a test statistic and we know it's distribution, do we need to know the asymptotic distribution (I mean doesn't all the distributions tend to have a normal one when the size of the sample tends to infinity?) Commented Mar 23, 2016 at 9:03
• You don't need an asymptotic distribution unless for some reason you can't use the exact one for some large sample size you have. Not all distributions tend to the normal, asymptotically. Commented Mar 23, 2016 at 9:07
• Thank you very much, you've been very helpful I apreciate it. One last thing, can you please reffer me to some articles that propose their own statistic tests? I want to see what do they proceed in order to come up with the test statistic and what do they doin order to know it's distribution. Commented Mar 23, 2016 at 9:30
• Well okay, but from the sound of your post they'll mostly be beyond you. 1. Here's a paper I was looking at recently which derives a way to approximate quantiles of the distribution of the intraclass correlation coefficient. (In the paper they use it for confidence intervals rather than hypothesis tests, but it could be used for a hypothesis test.) $\:$ ... ctd Commented Mar 23, 2016 at 10:34
• ctd... 2. Something simpler -- here's Pearson's 1900 paper in which he derives the chi-square distribution for a chi-square goodness of fit test (though he does already take the multivariate normal approximation to the multinomial as a given. However, you may want to read Plackett's paper Commented Mar 23, 2016 at 10:34