Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Assume a one-sample t-test, where the null hypothesis is $\mu=\mu_0$. The statistic is then $t=\frac{\overline{x}-\mu_0}{s/\sqrt{n}}$ using the sample standard deviation $s$. In estimating $s$, one compares the observations to the sample mean $\overline{x}$:

$s=\sqrt{\frac{1}{n-1}\sum_{i=1}^n (x_i-\overline{x})^2}$.

However, if we assume a given $\mu_0$ to be true, one could also estimate the standard deviation $s^*$ using $\mu_0$ instead of the sample mean $\overline{x}$:

$s^*=\sqrt{\frac{1}{n-1}\sum_{i=1}^n (x_i-\mu_0)^2}$.

To me, this approach looks more natural since we consequently use the null hypothesis also for estimating the SD. Does anyone know whether the resulting statistic is used in a test or know, why not?

share|improve this question

migrated from mathoverflow.net Jan 30 at 21:47

This question came from our site for professional mathematicians.

add comment

2 Answers 2

up vote 6 down vote accepted

There was a problem with the original simulation in this post, which is hopefully now fixed.

While the estimate of sample standard deviation tends to grow along with the numerator as the mean deviates from $\mu_0$, this turns out to not have all that big an effect on power at "typical" significance levels, because in medium to large samples, $s^*/\sqrt n$ still tends to be large enough to reject. In smaller samples it may be having some effect, though, and at very small significance levels this could become very important, because it will place an upper bound on the power that will be less than 1.

A second issue, possibly more important at 'common' significance levels, seems to be that the numerator and denominator of the test statistic are no longer independent at the null (the square of $\bar x-\mu$ is correlated with the variance estimate).

This means the test no longer has a t-distribution under the null. It's not a fatal flaw, but it means you can't just use tables and get the significance level you want (as we will see in a minute). That is, the test becomes conservative and this impacts the power.

As n becomes large, this dependence becomes less of an issue (not least because you can invoke the CLT for the numerator and use Slutsky's theorem to say than there's an asymptotic normal distribution for the modified statistic).

Here's the power curve for an ordinary two sample t (purple curve, two tailed test) and for the test using the null value of $\mu_0$ in the calculation of $s$ (blue dots, obtained via simulation, and using t-tables), as the population mean moves away from the hypothesized value, for $n=10$:

$\quad\quad\quad\quad$ n=10

enter image description here

You can see the power curve is lower (it gets much worse at lower sample sizes), but much of that seems to be because the dependence between numerator and denominator has lowered the significance level. If you adjust the critical values appropriately, there would be little between them even at n=10.

And here's the power curve again, but now for $n=30$

$\quad\quad\quad\quad$ n=30

enter image description here

This suggests that at non-small sample sizes there's not all that much between them, as long as you don't need to use very small significance levels.

share|improve this answer
add comment

When the null hypothesis is true, your statistic should be similar to the regular t-test statistic (though in computing the standard deviation you should probably divide by $n$ instead of $n-1$ because you are not spending a degree of freedom to estimate the mean). I would expect it to have similar properties (proper size, similar power) when the null hypothesis is true (the population mean is $\mu_0$.

But now consider what happens when the null hypothesis is not true. This means that in calculating the standard error you are subtracting a value that is not the true mean, or an estimate of the true mean, in fact you could be subtracting a value that does not even lie within the range of the x values. This will make your standard deviation larger ($\bar{x}$ is guaranteed to minimize the standard deviation) as $\mu_0$ moves away from the true mean. So when the null is false you will be increasing both the numerator and the denominator in the statistic which will reduce your chances of rejecting the null hypothesis (and it will not be distributed as a t-distribution).

So when the null is true either way will probably work, but when the null is false, using $\bar{x}$ will give better power (and probably other properties as well), so it is preferred.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.