Reliance of t-test upon particular estimate of standard deviation

Question

When we do the t-test, we typically take the estimate for standard deviation to be the square root of the usual unbiased estimate of variance, $s$. However, there are other ways of estimating standard deviation. One that comes to mind is the square root of the (biased) MLE for variance, where we divide by $n$ instead of $n-1$. Another is interquartile range divided by 1.35, which is robust to extreme observations.

Does it matter which standard deviation estimate we use or just that we have to estimate the unknown standard deviation?

Answer 1

I am presuming we're sticking with the assumption of normality under which the t distribution for the t-statistic is derived.

The derivation of the t-distribution relies on the specific form of the denominator, and particular facts about it (such as that it is distributed as the square root of a chi-squared random variable on its d.f., and that it's independent of the numerator).

1. $\frac{\bar{x}-\mu}{s_n/\sqrt n}$ is not distributed as a standard $t_{k}$ for any $k$; however, since $s_n = \sqrt\frac{n-1}{n} s$, it is a simple multiple of a standard $t_{n-1}$ (and thereby easy to deal with).

2. Let's consider $\tilde{s}=IQR/1.35$

   Note that $\tilde{s}^2/k$ is not distributed as chi-squared($k$), nor even as any multiple of it, though as sample size grows it gets reasonably close. However, it is the case that $\tilde{t} = \frac{\bar{x}-\mu}{c\tilde{s}}$ is approximately $t_\nu$ for some $c$ and $\nu$, but the particular values for those will change with sample size. With a little investigation it's possible to write an approximate t-test based on mean and IQR. The d.f. tend to be smaller than $n-1$

   For example, at $n=5$, $\tilde{t} = \frac{\bar{x}-\mu}{\tilde{s}/c}$ (using $\text{IQR}=X_{(4)}-X_{(2)}$) is approximately $t_{1.75}$ when $c$ is somewhere around $3.4$

   There's not a smooth relationship of $c$ and $\nu$ with $n$ because of the way sample IQR tends to related to either one or two order statistics at different sample sizes, but one can find smooth relationships when splitting the sample sizes into the four possible $n \mod 4$ values (exactly what they are depends on how you define your sample quartiles).

   It looks like the loss of d.f. in large samples is not substantial.

   Naturally, power will be lost at the normal, because $\sigma^2$ is not being efficiently estimated; the statistic is "noisier".