Can this be modeled by a two-sample t-test?

Question

Here is the question:

A claim is made that men from two income families earn less than men who are the sole income earners in their household:

To support this claim, data from a random sample of 22 married men was gathered. For this sample, the mean and standard deviation of salaries for 11 married men in two-income families was \$95,000 and \$15,000 respectively.

The mean and standard deviation for the remaining 11 married men who were the sole source of family income was \$124,000 and \$18,000 respectively.

(a) What are appropriate null and alternative hypotheses to test this claim?

(b) State an appropriate test statistic to test this hypothesis and the null distribution of this test statistic, stating any assumptions that you use.

(c) Calculate the test statistic chosen in (b), the corresponding p-value and form an appropriate conclusion.

Here is my answer:

(a) Essentially the null hypothesis is that there is no difference between married men from two income households versus one. So, I have:

$H_{0} : U_{x}=U_{y}$, the alternative is that men who are sole earners do earn more, $H_{1} : U_{x}<U_{y}$

(b) Since the pop. variance is unknown, I use a pooled variance - a test statistic following a t-distribution is used :

$\tau = \frac{\bar{X}-\bar{Y}}{S_{p}\sqrt{\frac{1}{n_{x}}+\frac{1}{n_{y}}}} \tilde{} t(n_{x}+n_{y}-2) = 20 $

My assumptions are that the samples have been obtained from a normal distribution population and that the populations have equal variance.

(c) For the men from two income households we have:

$\bar{x} = \$95,000 $ and $S_{x} = \$15,000 $, and the sole earners: $\bar{y} = \$124,000 $ and $S_{y} = \$18,000 $

The pooled variance is:

$s_{p}^2 =\frac{(n_{1} - 1)s_{1}^2 + (n_{2} - 1)s_{2}^2}{n_{1}+n_{2} -2} $

$ = \frac{(11 - 1)15,000^2 + (11 - 1)18,000^2}{11+11 -2} = \$274,500,000$

so $ s_{p}= \$16,584.04$

Therefore the observed test statistic is:

$\tau_{obs} = \frac{95,000-124,000}{16,568.04 \cdot \sqrt{\frac{2}{11}}} \approx -4.10$

With a 5% level of significance, the critical value is: $t_{0.05,20} = -1.725$

Since $-4.10 < -1.725$ we reject the null hypothesis.

The alternative hypothesis is $H_{1}: U_{x}-U_{y} < 0$, so the p-value is $P(t_{20} \le \tau_{obs}) = P(t_{20} \le -4.1) \approx 0.00028$

Since the p value is less than the level of significance we can conclude that we reject the null hypothesis at the 5% level of significance. I.e. the average salaries of men who are sole earners are higher than those men in two income homes.

Have I done this correctly? Also, what exactly is the 'null distribution' of this test statistic? I read somewhere that it is the distribution when the null hypothesis is true. So is that just the assumption that the samples are from a normal distribution? I don't quite understand this concept.

Many thanks for the feedback in advance!

Answer 1

This is a reasonable approach. Your calculations are correct numerically.

There's one caveat in the 'draw an appropriate conclusion' here. This is something of a perennial on here: "how to interpret a p-value".

The short answer is "it's $P(D|H)$" meaning the probability of the data given the hypothesis (which, naturally, was stated before performing the experiment). In your case, it's the probability of drawing this sample of 22 responses given your prior null hypothesis that the means are the same. Of course, here it's much $<5\%$. This doesn't translate as "sole earners earn more", which is $P(H|D)$. There's a more detailed explanation here..

The null distribution is just the distribution of the statistic of interest. In your case it's a $t$ distribution. You have calculated a statistic based on certain inputs (means and sds). Then check the probability that the value of this no. falls outside a certain range in the distribution of interest. In calculating the statistic you have made the additional assumption that the means and sds are figures which describe a normal distribution.

If using R you might like to verify things in an empirical case with

set.seed(1)
ms <- rnorm(11, mean=124, sd=18)
m2 <- rnorm(11, mean=95, sd=15)

then using debugonce(t.test) before

t.test(ms, m2, alternative="greater", var.equal=TRUE)

Better still look at the source with stats:::t.test.default; you can follow the logic, substitute exact rather than empiric values, and consider some additional ways to approach the problem.

Update Now this may seem like splitting hairs, but I think it's an important distinction which often gets overlooked in introductory stats courses. This $t$-test assumes the frequentist worldview. You have made certain assumptions (samples really are normally distributed, equal variance, means are the same) in generating a statistic. The probability of drawing such a statistic from the $t$ distribution is indeed very low. Hence the probability of observing such data as lead to the statistic is indeed very low. You can indeed use such a finding to "support a claim" and this is widespread in practice. Strictly speaking however, justifying a statement such as "sole earners earn more" requires that you approach the given data from a Bayesian perspective...