5
$\begingroup$

I am doing multiple logistic regression with data with 24 predictor variables and 193 rows. All predictor variables have values of 0 or 1 and outcome variables (OUTVAR) also has only 2 possibilities.

I am using following code:

import statsmodels.discrete.discrete_model as sm
model = sm.Logit.from_formula(formula=formulastr, data=df)
model_fit = model.fit()
print(model_fit.summary()) 

The results are as follows:

                           Logit Regression Results                           
==============================================================================
Dep. Variable:                 OUTVAR   No. Observations:                  193
Model:                          Logit   Df Residuals:                      167
Method:                           MLE   Df Model:                           25
Date:                Sun, 15 Dec 2019   Pseudo R-squ.:                  0.4691
Time:                        19:58:12   Log-Likelihood:                -60.734
converged:                      False   LL-Null:                       -114.40
Covariance Type:            nonrobust   LLR p-value:                 3.546e-12
==========================================================================================
               coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------------------
Intercept   -7.1429      1.398     -5.109      0.000      -9.883      -4.403
var1         0.0359      1.001      0.036      0.971      -1.926       1.998
var2         0.2542      0.630      0.403      0.687      -0.981       1.489
var3         0.9039      0.670      1.350      0.177      -0.408       2.216
var4         0.3396      0.632      0.538      0.591      -0.898       1.578
var5         0.3985      1.077      0.370      0.711      -1.712       2.509
var5         0.1168      1.101      0.106      0.916      -2.041       2.275
var6         1.6755      0.566      2.961      0.003       0.566       2.785
var7         0.7480      0.716      1.045      0.296      -0.655       2.151
var8        22.9672  12194.967      0.002      0.999  -23878.729   23924.663
var9        -0.7337      1.020     -0.720      0.472      -2.732       1.265
var10        1.8130      0.983      1.844      0.065      -0.114       3.740
var11       -0.1299      0.619     -0.210      0.834      -1.344       1.084
var12        0.7897      0.571      1.383      0.167      -0.329       1.909
var13        0.0465      0.680      0.068      0.946      -1.286       1.379
var14       -0.7226      0.573     -1.262      0.207      -1.845       0.400
var15        0.9850      0.571      1.724      0.085      -0.135       2.105
var16        0.3825      0.578      0.662      0.508      -0.751       1.516
var17        0.6759      0.595      1.137      0.256      -0.489       1.841
var18        1.4240      0.556      2.559      0.010       0.333       2.515
var19        0.1379      0.661      0.209      0.835      -1.157       1.433
var20        2.3520      1.060      2.219      0.026       0.275       4.429
var21       -0.5318      0.694     -0.766      0.443      -1.892       0.828
var22        0.3063      0.582      0.526      0.599      -0.835       1.448
var23        1.3203      0.661      1.996      0.046       0.024       2.616
var24       -0.1218      0.848     -0.144      0.886      -1.783       1.540
==========================================================================================

My question is what could be the reason large std error (hence also confidence interval range) for var8 to be so large as compared with all other variables? What does this kind of result mean? Also, can we conclude that only var6, var18, var20 and var23 are independently related to OUTVAR and all others are not significantly related?

Edit: In response to some comments:

* Number of iterations: 35
* var8 is correlated with outcome variable: P<0.0001
* var8 is not highly correlated with any other predictor variable: maximum R is 0.22

However, var8 does completely separates:

OUTVAR      No  Yes
var8          
No         139   47
Yes          0    7

So that must be the reason for large (but insignificant) result in multiple regression.

$\endgroup$

1 Answer 1

8
$\begingroup$

You have too many variables for the amount of data you have. A rough rule of thumb is that, in logistic regression, you can have $1$ predictor variable for every $15$ observations in the less commonly occurring category. With $193$ data, you can have at most $97$ instances of yeses or noes. That implies you should use no more than $6$ predictors.

That reasoning relates to all variables. Regarding that specific variable as distinct from the others, it is presumably either collinear or completely separates the yeses or noes. To diagnose which might lie behind this, see how many fitting iterations were used, $>10$ is some evidence of separation; and fit an ordinary least squares multiple regression of var 8 against the all rest (it doesn't matter if the assumptions are met) and check the multiple $R^2$, a value $>.9$ considered problematic. To understand separation, it may help to read my answer here: Why does logistic regression become unstable when classes are well-separated? To understand multicollinearity, it may help to read my answers here: What is the effect of having correlated predictors in a multiple regression model?, and here: How seriously should I consider the effects of multicollinearity in my regression model?

Regarding your last question of whether you can conclude that the null holds for the non-significant variables, it might help you to read my answer here: Why do statisticians say a non-significant result means “you can't reject the null” as opposed to accepting the null hypothesis?

$\endgroup$
8
  • $\begingroup$ The outcome variable categories are distributed as: 54 and 139. How many variables can I have? $\endgroup$
    – rnso
    Commented Dec 15, 2019 at 15:10
  • $\begingroup$ @rnso, remember that it's just a rule of thumb, not a law of nature, but that would suggest you try to limit it to 4. $\endgroup$ Commented Dec 15, 2019 at 15:15
  • 1
    $\begingroup$ @gung-ReinstateMonica I had always heard/read that the rule of thumb was 10 observtions in the less common category, which would mean 5 here. I don't have a cite for the 10 .... Do you have one for 15? $\endgroup$
    – Peter Flom
    Commented Dec 15, 2019 at 15:51
  • 1
    $\begingroup$ For some ideas on what to do about complete or quasi-complete separation, see onlinelibrary.wiley.com/doi/10.1002/0471475769.ch10 $\endgroup$
    – Peter Flom
    Commented Dec 15, 2019 at 15:54
  • 1
    $\begingroup$ @PeterFlom-ReinstateMonica, Frank Harrell has said this (eg, see: Sample size for logistic regression?). $\endgroup$ Commented Dec 15, 2019 at 18:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.