Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a problem which examines a two independent sample t-test with unequal variances. The null hypothesis is $\mu_1=\mu_2$ and the alternative the opposite. As the problem is stated it turns out that the null hypothesis is accepted. The question is, how much more should the sample sizes be increased in order to reject the null hypothesis? Is there a standard way to find the sample size of each sample or should I find a relationship between them?

($t_{0.025,79}$ = 1.990, t-score = 1.5830.)

Thank you in advance.

share|improve this question
There's no way to solve the problem without providing 1) the true effect size (μ1-μ2). If it is large, you won't need a large sample size, and if it is small, you will, and if μ1=μ2, then no sample size will say it is significant. 2) What probability should it have to reject the null hypothesis? It is always possible that you won't reject the null hypothesis, you just want to specify the probability that you will correctly reject it. (This is called the power of the test). – David Robinson Dec 12 '12 at 21:41
the question is how many more observations should i have in order to have statistical significant difference, assuming that after the addendum of the new observations the mean and std of each distribution does not change .. any ideas? – user1681361 Dec 12 '12 at 21:51
So you're saying you already have some data, and you're assuming the estimated means are the true means. That's dangerous- you know that they're not the true means, and that if you ran a different experiment you would get different means (potentially in either direction). I take it that you ran an experiment and you want to know how large you should make the next one? – David Robinson Dec 12 '12 at 21:56
exactly, that's what i want – user1681361 Dec 12 '12 at 22:03
This question has a bunch of problems: 1) you never accept the null, you just sometimes fail to reject it 2) Increasing the sample size makes it more likely to reject the null, not fail to do so 3) As @DavidRobinson said, it's like cheating. – Peter Flom Dec 12 '12 at 22:05

You're interested in the power of the t-test, given a certain effect size and standard deviation (that you've estimated with a smaller experiment).

In R, this can be calculated with the power.t.test function. For example, if you've estimated that the true difference between the means is .6 and the standard deviation is .5, you could do:

plot(x, power.t.test(x, delta=.6, sd=.5)$power, type="l")

enter image description here

This shows the probability of the null hypothesis being rejected at each size. Note, however, that this will change dramatically if you've overestimated the effect size based on your first experiment.

share|improve this answer

There's an argument to be made that there isn't any amount of subjects you can add to reject the null assuming you want to maintain the same alpha level you started with. You've already done your test on your experiment. Assuming you used an alpha of 0.05 to select your cutoff, another test of additional data in this experiment will have an alpha > 0.05. There are no numbers of subjects that change that.

What you can do is get the effect size and variance in this experiment and use a program like G*Power or the command power.t.test in R to work out how many subjects you would need for a subsequent experiment in which you tested the effect again.

You really need to examine your effect size and confidence interval of your effect at this point, both reported in many t-test programs. Is the range of effects captured by your confidence interval narrow enough? If so, then you stop and say that there really isn't any meaningful difference between your conditions. If the confidence interval is very wide then you may collect data to make it narrower to make a stronger statement about what the effect actually is. But you cannot ever get alpha back down to 0.05 without a new experiment.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.