2
$\begingroup$

I have a problem which examines a two independent sample t-test with unequal variances. The null hypothesis is $\mu_1=\mu_2$ and the alternative the opposite. As the problem is stated it turns out that the null hypothesis is accepted. The question is, how much more should the sample sizes be increased in order to reject the null hypothesis? Is there a standard way to find the sample size of each sample or should I find a relationship between them?

($t_{0.025,79}$ = 1.990, t-score = 1.5830.)

Thank you in advance.

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ There's no way to solve the problem without providing 1) the true effect size (μ1-μ2). If it is large, you won't need a large sample size, and if it is small, you will, and if μ1=μ2, then no sample size will say it is significant. 2) What probability should it have to reject the null hypothesis? It is always possible that you won't reject the null hypothesis, you just want to specify the probability that you will correctly reject it. (This is called the power of the test). $\endgroup$
    – David Robinson
    Commented Dec 12, 2012 at 21:41
  • $\begingroup$ the question is how many more observations should i have in order to have statistical significant difference, assuming that after the addendum of the new observations the mean and std of each distribution does not change .. any ideas? $\endgroup$
    – user1681361
    Commented Dec 12, 2012 at 21:51
  • $\begingroup$ So you're saying you already have some data, and you're assuming the estimated means are the true means. That's dangerous- you know that they're not the true means, and that if you ran a different experiment you would get different means (potentially in either direction). I take it that you ran an experiment and you want to know how large you should make the next one? $\endgroup$
    – David Robinson
    Commented Dec 12, 2012 at 21:56
  • $\begingroup$ exactly, that's what i want $\endgroup$
    – user1681361
    Commented Dec 12, 2012 at 22:03
  • 3
    $\begingroup$ This question has a bunch of problems: 1) you never accept the null, you just sometimes fail to reject it 2) Increasing the sample size makes it more likely to reject the null, not fail to do so 3) As @DavidRobinson said, it's like cheating. $\endgroup$
    – Peter Flom
    Commented Dec 12, 2012 at 22:05

2 Answers 2

Reset to default
2
$\begingroup$

You're interested in the power of the t-test, given a certain effect size and standard deviation (that you've estimated with a smaller experiment).

In R, this can be calculated with the power.t.test function. For example, if you've estimated that the true difference between the means is .6 and the standard deviation is .5, you could do:

plot(x, power.t.test(x, delta=.6, sd=.5)$power, type="l")

enter image description here

This shows the probability of the null hypothesis being rejected at each size. Note, however, that this will change dramatically if you've overestimated the effect size based on your first experiment.

Improve this answer
$\endgroup$
0
$\begingroup$

There's an argument to be made that there isn't any amount of subjects you can add to reject the null assuming you want to maintain the same alpha level you started with. You've already done your test on your experiment. Assuming you used an alpha of 0.05 to select your cutoff, another test of additional data in this experiment will have an alpha > 0.05. There are no numbers of subjects that change that.

What you can do is get the effect size and variance in this experiment and use a program like G*Power or the command power.t.test in R to work out how many subjects you would need for a subsequent experiment in which you tested the effect again.

You really need to examine your effect size and confidence interval of your effect at this point, both reported in many t-test programs. Is the range of effects captured by your confidence interval narrow enough? If so, then you stop and say that there really isn't any meaningful difference between your conditions. If the confidence interval is very wide then you may collect data to make it narrower to make a stronger statement about what the effect actually is. But you cannot ever get alpha back down to 0.05 without a new experiment.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.