How can I perform a two sample test of means with unequal variances for a very large sample in R?
In case of large samples the statistic will asymptotically follow a normal distribution.
Which R function will help me to do this?
$\begingroup$
$\endgroup$
$\begingroup$
$\endgroup$
While you can compute the z-statistic, actually an ordinary Welch t-test will do that just fine - in R that's t.test with all its default options.
The form of test statistic is the same in both cases. The only difference is in which table is used, and if the size of the smaller group is large enough, the tests will give almost identical p-values.
The Welch test will handle very large sample sizes.
e.g. in R:
> x=rnorm(1e7,1.00001,1)
> y=rnorm(1e7,1.00002,2)
> t.test(x,y)
Welch Two Sample t-test
data: x and y
t = 0.9052, df = 14708415, p-value = 0.3654
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-0.0007458214 0.0020259201
sample estimates:
mean of x mean of y
0.999757 0.999117
I don't see a problem
> # compare:
> 2*pnorm((-abs(mean(y)-mean(x))/sqrt(var(y)/length(y)+var(x)/length(x))))
[1] 0.3653657
The p-values turn out to be the same to all the places shown in the second figure.
If that's not what you want, you need to more carefully explain what you do want.
Example with very different $n$:
> x=rnorm(1e7,1.00001,1)
> y=rnorm(1e2,1.002,2)
> t.test(x,y)
Welch Two Sample t-test
data: x and y
t = 0.7382, df = 99.001, p-value = 0.4622
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-0.2409398 0.5264124
sample estimates:
mean of x mean of y
0.9998066 0.8570703
> 2*pnorm((-abs(mean(y)-mean(x))/sqrt(var(y)/length(y)+var(x)/length(x))))
[1] 0.4604087
Once we're at 99df for the Welch, we start to notice a small difference in p-value from the asympotic result, but since we're at 99d.f., we're not really in the 'consider it as converged to normal' region.
-
$\begingroup$ I think this should do it. So, even if sample sizes of x and y are different there won't be a problem right? $\endgroup$ – darkage Aug 1 '14 at 4:38
-
$\begingroup$ It depends on just how different they are and how different the variances are -- but if things are such that the t and the z (see the second calculation) don't give the same p-value, I'd be more leery of the z than the t. The Welch-Satterthwaite d.f. should tell you if that's an issue. If that's in the hundreds, it won't make a practical difference unless you're well into the tail. $\endgroup$ – Glen_b Aug 1 '14 at 4:42
-
-
$\begingroup$ I don't know what you're after. Could you expand on what you want expanded on? (NB added another example with very different $n$) $\endgroup$ – Glen_b Aug 1 '14 at 4:51
-
$\begingroup$ I've got what I was looking for. Just want to know how will d.f. help me to know what is the issue. What is the concept behind this $\endgroup$ – darkage Aug 1 '14 at 4:54
$\begingroup$
$\endgroup$
A few points in addition to what @Glen_b makes.
1) The default in t.test is var.equal = FALSE which means it assumes the samples have different variances.
2) I think it is a useful exercise to just skim over the proof of this result in the case of the z-test (involves Slutsky's theorem and the CLT) which you will find online quite easily. Even if all the steps don't make sense (like convergence in distribution for example) it will give you a good sense for why you really care about these degrees of freedom.
