2
$\begingroup$

I have got 2 large samples and want to compare the difference. There are 2 methods:

  1. Assume the population variances are unknown and unequal, and use the Welch $t$ test.
  2. Assume the population variances are known and unequal (estimated as sample variances, which is ok since my sample size is million level and the sample mean can be considered as normally distributed), and use the $Z$ test.

I found these 2 tests are pretty much the same formula and same result. So are these 2 tests the same when applied to the real world situation? And if not, what is the difference and when should I use the $Z$ test/welch $t$ test?

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ Welch's test relates to the Behrens-Fisher problem . Fisher posed a distribution based on is controversial fiducial inference. Welch provided a test that works well under frequentist methods. The problem is that the usual t statistic doesn't have a t distribution because both variances are unknown and assumed to be unequal. The sample is assumed to be from two normal distributions. Welch approximated the t distribution with a t that has a fractional number for the degrees of freedom. $\endgroup$
    – Michael R. Chernick
    Dec 13 '16 at 2:42
4
$\begingroup$

Correct, Welch is for unequal variances. $Z$-test is for large-sample approximation when the standard normal distribution approximates the $t$-distribution. For sample sizes above about $n=130$, you will find that $Z$-scores are close to $t$-variates. The $t$-distribution is not needed when there are hundreds of objects (or records), since the standard normal distribution approximates the $t$ distribution very well for large $n$.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I am not sure what rule of thumb makes 130 appropriate. In very large samples Welch's t closely approximates the t which closely approximates the normal but I think it is important to recognize that the exact distribution is not exactly a t. $\endgroup$
    – Michael R. Chernick
    Dec 13 '16 at 2:47
  • $\begingroup$ under what kind of situation, only 1 of these 2 test is suitable? $\endgroup$
    – user3516947
    Dec 13 '16 at 21:25
  • $\begingroup$ Uneven sample sizes ($n_1$ vs. $n_2$) I recall is an issue when comparing the t-test for unequal variances with the Welch test. $\endgroup$
    – user32398
    Dec 14 '16 at 0:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.