1
$\begingroup$

I am trying to do some significnce testing using Python statsmodels.

I have compiled two tests using OLS and weightstats.ttest_ind on the same data. My dataset contains one independent variable with values 0 and 1 and a continuous dependent variable. The number of 0 and 1 values is unequal which is why I used the option usevarstr = ‘unequal’ for the t-test. This yielded different outcomes between OLS and t-test. Running the t-test with usevarstr = ‘pooled’ however gave me the same results as OLS, except for the p-value. I do not understand why the p-values is so much higher in the t-test.

ols

t-test with usevarstr='pooled': (4.864087195854719, 1.1864780944353952e-05, 50.0)

t-test with usevarstr='unequal': (4.8062218093574405, 1.7557614217538848e-05, 44.90172116268066)

Is there an option in statsmodels OLS that is equivalent to t-test with unequal distribution? And if not, can I rely on the results I am getting now although they assume equal distribution (as far as I understand it)?

Or maybe there is something esle going on that I am missing?

$\endgroup$
1
$\begingroup$

usevarstr has to do with the variance of the two groups, not the sample sizes. This makes sense. The computer doesn’t need you to tell it if the sample sizes are uneven; it has the samples and can determine that itself. However, it does not know if you’re assuming equal or unequal population variances in the two groups until you tell it.

When you set usevarstr = ‘unequal’, you are saying to do a Welch t-test that assumes unequal population variances. When you set ‘pooled’, you are saying to do the usual t-test that pools the variances of each group to estimate the equal variance of the two populations.

Consequently, ‘pooled’ yields the same results as the OLS that assumes normal error terms with equal variances.

Since you only have one variable (plus the intercept parameter) the Prob(F-statistic) is the p-value on the parameter for your group variable. It just gets rounded down to zero in the chart below.

$\endgroup$
3
  • $\begingroup$ Thank you for this clarification! Assuming that the variance is unequal would it be OK to use OLS? Or is there any equivalent of usevarstr = ‘unequal’ in OLS? I found the option varcorrection_pairs_unbalanced in statsmodels, which is not public yet. Would that be an equivalent of usevarstr = ‘unequal’? $\endgroup$ – user9397006 Apr 7 '20 at 11:01
  • $\begingroup$ Generalized least squares would be more appropriate in that case, though, for uneven sample sizes, my memory is saying that it’s not quite the same as the Welch t-test. $\endgroup$ – Dave Apr 7 '20 at 13:04
  • $\begingroup$ Welch t-test is equivalent to OLS with cov_type='HC2', except that OLS doesn't have the Welch-Satterthwaite degrees of freedom correction, i.e. same test statistic but different p-value. $\endgroup$ – Josef Apr 7 '20 at 13:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.