2
$\begingroup$

I've been asked to analyze some data but I'm quite new in statistic. I want to compare the means of a continuous parameter in two independent population A and B: A can be seen as the treatment group whereas B can be seen as the control group. I have two samples, for population A and for population B, whose dimensions are 79 and 15 respectively. I would like to do a hypothesis test with null hypothesis $H_0: \mu_A=\mu_B$ (I haven't seen the data yet, but I expect to reject $H_0$). Obviously I will use the sample mean $\overline{X}_A$ as estimator for $\mu_A$ and $\overline{X}_B$ as estimator for $\mu_B$, $H_0$ can be written as $H_0: \mu_A-\mu_B=0$, so the estimator I have to use for $\mu_A-\mu_B$ is $\overline{X}_A-\overline{X}_B$

  1. The variance of $\overline{X}_A$ and of $\overline{X}_B$ are unkown, so I have to use the sample standard deviations $s_A$ and $s_B$, hence I extimate the standard deviation of $\overline{X}_A-\overline{X}_B$ with $\sqrt{s_A^2+s_B^2}$. I use a t-test with $$t=\frac{\overline{X}_A-\overline{X}_B}{\sqrt{s_A^2+s_B^2}}.$$ The question is: why do I use a t-statistic? Is it correct? I read that, when the standard deviation is unkown, the t-statistic is appropriate provided that $\overline{X}_A-\overline{X}_B$ is normally distributed... But I have no idea if $\overline{X}_A-\overline{X}_B$ is normally distributed.

  2. Is it ok the proportion between the dimensions of sample A and B? We have 79 vs 15 so 84% vs 16%. What are the problems I can run into if the proportion is not appropriate?

  3. For example, given that using a t-test as above is correct, if I fix the probability of a I type error of 5%, can I be sure that it is a real 5% or is it bigger? What about the power of the test? I expect to reject $H_0$, so I hope my test is powerful enough in order to do that.

  4. If this partition (84|16) is not good, is increasing the dimension of the control group the only way to make a more significant statistic?

Any suggestion or good reference will be more than welcome.

|cite|improve this question
$\endgroup$
3
$\begingroup$

The basic assumptions of the two-sample t-test are that the distributions of the populations are approximately normal and that their variances are the same. There is no requirement that the sizes of the groups be the same, however if the variances are different, it is only possible to proceed if the sample sizes are roughly equal, which is not the case here.

So, in your case, provided that the variances of the two groups are approximately equal, and both samples are approximately normally distributed, you can perform a t-test.

To test for the equality of variance you could use Levine’s test (or an F test)

To test for normality formally you could use the Shapiro-Wilk test (or the Anderson-Darling test or the Kolmogorov-Smirnov test), but first it is a good idea to assess normality informally using a QQ plot and histogram. It is OK that the groups are of unequal size – provided that the variances are approximately equal, although the test would certainly have more power if the smaller group was bigger - in fact, power is maximised when the group sizes are the same.

You ask “why do I use a t-statistic” ? I’m not sure how to answer that, other than saying “because you are doing a t-test and the sampling distribution of the test statistic follows a t-distribution with the relevant degrees of freedom, under the null hypothesis.”

If your data are obviously not normally distributed (and can’t be transformed) then you can use a non-parametric two-sample test such as the Mann-Whitney U test (Wilcoxon two-sample test) If the variances are obviously unequal (but the data are plausibly normal) you can consider other tests such as the Welch-Satterthwaite T test

|cite|improve this answer
$\endgroup$
  • $\begingroup$ So: 1.check normality in population and check equality of variances. If 1. is ok then do a standard t-test. 2. If the populations ARE normal and the variances are different, then do a Welch test. 3. If the population is not normal and the variances are not equal, then do a Mann-Whitney test (I don't know this test, but googling it, it seems that it works if assumptions in 3. are met). Instead of checking 1. and 2., can't we just do Mann-Whitney test directly? If the variances are not equal, do we use Mann-Whitney test freewheeling even if the group sizes are not equal? Thank you very much $\endgroup$ – Lisa Nov 29 '18 at 17:34
  • 1
    $\begingroup$ You can do MW instead of Student's t test, but if both assumptions are satisfied the t-test should have more power. You can also do both tests. $\endgroup$ – Robert Long Nov 29 '18 at 17:41
  • $\begingroup$ Do you know some references where I can study what happen if the sample size is significantly different? $\endgroup$ – Lisa Nov 29 '18 at 17:50
  • $\begingroup$ Wait, accordingly to wikipedia "Under the null hypothesis H0, the distributions of both populations are equal"... But if they don't have equal variances then they don't have equal distribution hence we can't use MW test. What sould one do, then? $\endgroup$ – Lisa Nov 29 '18 at 18:57
  • $\begingroup$ @Lisa see here $\endgroup$ – Robert Long Nov 29 '18 at 19:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.