8
$\begingroup$

I have a question on reading t-test p-value. If I understand correctly, the t-statistic is computed as:

$t = \frac{\bar{X} - \mu}{\sigma / \sqrt{n}}$

where $\bar{X}$ is sample mean, $\mu$ is population mean, $\sigma$ is sample standard deviation and $n$ is size of sample. Degrees of freedom is $n-1$.

Now, we can see that the t-statistic is inversely proportional to the standard error/variance of the sample population ($\sigma / \sqrt{n}$). Higher $n$ leads to smaller standard error that gives higher t-value. Higher t-value means lower p-value infering that the difference between sample-mean ($\bar{X}$) and population-mean ($\mu$) is significant (hence we reject the null hypothesis).

But this formula seems counter-intuitive to me as bigger sample size (higher $n$) should give sample mean closer to population mean.

How do we explain this?

$\endgroup$
1
  • $\begingroup$ In what sense do you mean "increase with sample size?" After all, $\bar{X}$ changes with sample size, too: you can't hold it fixed while increasing $n$. To put it another way: $t$ is a random variable. Therefore, for any fixed population, you obtain a sequence of random variables, one for each $n$. Could you clarify what you mean for this sequence of random variables to be "increasing"? $\endgroup$
    – whuber
    Jul 31, 2011 at 19:49

3 Answers 3

6
$\begingroup$

A little change in notation might help in answering your question: What you call $\mu$ is often called $\mu_{0}$ because it is the population mean under the null hypothesis, whereas $\mu$ is the actual population mean - which is unknown because we don't know whether the null hypothesis is true. Also, what you call $\sigma$ is usually called $s$ following the convention that population parameters get greek letters, and sample parameters are denoted by latin letters.

Note that $s / \sqrt{n}$ is the standard error of the mean $\bar{X}$ (SEM) - an estimate of the variablity of $\bar{X}$, where $\bar{X}$ is understood as a random variable. So we have

$t = \frac{\bar{X} - \mu_{0}}{s / \sqrt{n}}$

Now, for a given sample, you have a fixed empirical $\bar{X}_{emp}$, and thus a fixed difference $d_{emp} = \bar{X}_{emp} - \mu_{0}$. Part of the confusion seems to be related to the idea that "a bigger sample size (higher $n$) should give sample mean closer to population mean". This should be rephrased to conditional on the null hypothesis being true ($\mu = \mu_{0}$), the probability of observing a difference $d = \bar{X} - \mu_{0}$ that is at least as large as the already observed $d_{emp}$ becomes smaller when $n$ increases. This is because the "accuracy" of our estimator $\bar{X}$ then increases (variability decreases).

I guess the main point is that you already have a fixed $\bar{X}_{emp}$ and thus $d_{emp}$, and $t$ just tells you how "big that difference is" measured in (estimated) units of variability of $\bar{X}$. When the units become smaller in absolute numbers, the same absolute difference $d_{emp}$ will "be worth more units" and will thus count as "more suprisingly high" (= less likely to occur) if $\mu = \mu_{0}$.

$\endgroup$
1
  • 1
    $\begingroup$ Thanks for the sound explanation. I was confusing myself with samples selection but now it is clear. $\endgroup$
    – user862
    Aug 6, 2011 at 11:22
2
$\begingroup$

Someone else can probably give a more rigorous answer, but:

For any given (fixed) difference between $\bar{X}$ and $\mu$, the difference is more meaningful if n is high.

Increasing n will result in a sample mean closer to the population mean, but only in the case that your sample is not different from the population. So when n is high and $\bar{X}$ still differs from $\mu$, that reinforces the rejection of the null hypothesis.

$\endgroup$
2
  • $\begingroup$ That's an interesting explanation. However, I still don't understand when you have two samples of different sizes with the same difference in mean from population mean, why do we penalize the larger sample? $\endgroup$
    – user862
    Jul 31, 2011 at 18:12
  • 1
    $\begingroup$ Hmm - I think you actually 'reward' the one with bigger n, because $t= \frac{\mu-\mu_0}{S/{\sqrt{N}}} = \frac{\sqrt{N}(\mu-\mu_0)}{S}$ Yah? $\endgroup$
    – ImAlsoGreg
    Jul 31, 2011 at 22:19
0
$\begingroup$

Your last sentence seems to capsulize the confusion.

You wrote

But this formula seems counter-intuitive to me as bigger sample size (higher n) should give sample mean closer to population mean.

but this is true only if the sample is from a population that has the same mean as the population it is being compared to. The word "population" is being used to refer to two different populations

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.