I have two samples from two populations, and I want to test the null hypothesis that the means of these populations are equal vs the alternative that they're not equal. Suppose the populations are both $N(\mu_i, 2^2)$ distributed with variance of two and some possibly different, unknown means. All samples are independent.

Suppose our two population samples are as follows:

pop1 = c(5.14, 4.05, -0.511, 0.44, 3.27, 0.970, 6.03, 2.21, 3.93, 5.65)
pop2 = c(3.86, 3.37, 4.40, 9.69, 1.69, 6.06, 7.13, 3.45, 5.98, 8.65)  


data <- c(rnorm(10, 3, 2), rnorm(10, 4, 2))

Now, I think the standard thing to do here would be to conduct an unpaired t-test, but the following method makes some sense to me, and I'd like to know why it doesn't work.

  1. We know that the sample means of each population is distributed as $N(\mu_i,\frac{2^2}{n})$ where n is our sample size, in this case 10.

  2. Therefore, the statistic $\frac{x-\mu_h}{sqrt(\frac{2^2}{10})}$ is standard normally distributed under the null hypothesis.

    (Note: x is x-bar, the sample mean for the first population, and $\mu_h$ is mu-hat, the MLE under the null hypothesis for the mean of the populations, or the pooled sample mean).

  3. We then calculate a p-value by looking up a corresponding value in a standard normal table.

This method gives fairly different p-values to a t-test, and I'm wondering what the issues are with it. The more the merrier :)

  1. Your "$\mu_h$" is not a population quantity but a sample quantity ($\hat{\mu}_h$). The denominator of your statistic doesn't properly account for standard error of the numerator.

  2. With these sample sizes, your $\hat{\mu}_h$ is $\frac{\bar{x}_1+\bar{x}_2}{2}$, so your numerator $\bar{x}_1-\hat{\mu}_h$ is in fact $\frac{\bar{x}_1-\bar{x}_2}{2}$, exactly half the numerator of the ordinary t-statistic.

Once you correctly calculate the standard error of that numerator and put it on the denominator (the factor of $\frac12$ in the numerator will be cancelled by a corresponding factor of $\frac12$ in the denominator), ... you'll get something that has the same value as a two-sample t-statistic. You will in the end just be doing a plain old two-sample t-test.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.