In a one sample t-test, what happens if in the variance estimator the sample mean is replaced by $\mu_0$?

Question

Assume a one-sample t-test, where the null hypothesis is $\mu=\mu_0$. The statistic is then $t=\frac{\overline{x}-\mu_0}{s/\sqrt{n}}$ using the sample standard deviation $s$. In estimating $s$, one compares the observations to the sample mean $\overline{x}$:

$s=\sqrt{\frac{1}{n-1}\sum_{i=1}^n (x_i-\overline{x})^2}$.

However, if we assume a given $\mu_0$ to be true, one could also estimate the standard deviation $s^*$ using $\mu_0$ instead of the sample mean $\overline{x}$:

$s^*=\sqrt{\frac{1}{n-1}\sum_{i=1}^n (x_i-\mu_0)^2}$.

To me, this approach looks more natural since we consequently use the null hypothesis also for estimating the SD. Does anyone know whether the resulting statistic is used in a test or know, why not?

Answer 1

There was a problem with the original simulation in this post, which is hopefully now fixed.

While the estimate of sample standard deviation tends to grow along with the numerator as the mean deviates from $\mu_0$, this turns out to not have all that big an effect on power at "typical" significance levels, because in medium to large samples, $s^*/\sqrt n$ still tends to be large enough to reject. In smaller samples it may be having some effect, though, and at very small significance levels this could become very important, because it will place an upper bound on the power that will be less than 1.

A second issue, possibly more important at 'common' significance levels, seems to be that the numerator and denominator of the test statistic are no longer independent at the null (the square of $\bar x-\mu$ is correlated with the variance estimate).

This means the test no longer has a t-distribution under the null. It's not a fatal flaw, but it means you can't just use tables and get the significance level you want (as we will see in a minute). That is, the test becomes conservative and this impacts the power.

As n becomes large, this dependence becomes less of an issue (not least because you can invoke the CLT for the numerator and use Slutsky's theorem to say than there's an asymptotic normal distribution for the modified statistic).

Here's the power curve for an ordinary two sample t (purple curve, two tailed test) and for the test using the null value of $\mu_0$ in the calculation of $s$ (blue dots, obtained via simulation, and using t-tables), as the population mean moves away from the hypothesized value, for $n=10$:

$\quad\quad\quad\quad$ n=10

You can see the power curve is lower (it gets much worse at lower sample sizes), but much of that seems to be because the dependence between numerator and denominator has lowered the significance level. If you adjust the critical values appropriately, there would be little between them even at n=10.

And here's the power curve again, but now for $n=30$

$\quad\quad\quad\quad$ n=30

This suggests that at non-small sample sizes there's not all that much between them, as long as you don't need to use very small significance levels.

Answer 2

When the null hypothesis is true, your statistic should be similar to the regular t-test statistic (though in computing the standard deviation you should probably divide by $n$ instead of $n-1$ because you are not spending a degree of freedom to estimate the mean). I would expect it to have similar properties (proper size, similar power) when the null hypothesis is true (the population mean is $\mu_0$.

But now consider what happens when the null hypothesis is not true. This means that in calculating the standard error you are subtracting a value that is not the true mean, or an estimate of the true mean, in fact you could be subtracting a value that does not even lie within the range of the x values. This will make your standard deviation larger ($\bar{x}$ is guaranteed to minimize the standard deviation) as $\mu_0$ moves away from the true mean. So when the null is false you will be increasing both the numerator and the denominator in the statistic which will reduce your chances of rejecting the null hypothesis (and it will not be distributed as a t-distribution).

So when the null is true either way will probably work, but when the null is false, using $\bar{x}$ will give better power (and probably other properties as well), so it is preferred.