1
$\begingroup$

I am using sm.MLogit in python (i.e. multiclass logistic regression) to classify a 3 classes (0,1,2). I have numerical dependent variables that are standardized (mean 0, standard dev. 1) and I add an intercept (only) if it is significant according to the model.

This library uses a latent class. I see that one model is developed for class 0 and other for class 1, so the latent variable is 2. Bellow is the example output (deleted some parts, that I though were irrelevant for my question) of a model without intercept:

Model:              MNLogit          Pseudo R-squared: 0.364     
----------------------------------------------------------------
  label = 0  Coef.  Std.Err.   t    P>|t|  
----------------------------------------------------------------
first      0.6     0.3       2.1483 0.0317  
max        4.2     0.8       5.3240 0.0000 
delta      0.2     0.4       0.5482 0.5835 
----------------------------------------------------------------
 label = 1  Coef.  Std.Err.    t    P>|t|  
----------------------------------------------------------------
first     -0.3   0.3        -1.1453 0.2521 
max       5.1   0.8          6.4812 0.0000 
delta     0.7   0.3          2.1784 0.0294  
=================================================================

So, I interpret the coefficients for first as: An increase of one s. dev. on first will increase the odds of observing 0 over 2 by 80% [assuming odds ratio of exp(0.6)~1.8]. Likewise an increase of one s. dev. on first will decrease the odds of observing 1 over 2 by 30% [assuming odds ratio of exp(-0.3)~0.7].

This does not go along with what I see on a box plot of the variable first across classes: where 0 is in average lower than 2, that is lower than 1.

I use the same logic for sm.Logit (i.e. binary logistic regression in python) for binary classification (0,1), assuming then that the coefficients are for class 0 in reference to class 1, but the interpretation is not in accordance to boxplots of the variables, either.

$\endgroup$

1 Answer 1

0
$\begingroup$

I will answer my own question given that I could solve the issue, and the problem was not related to coefficient interpretation.

In short, removing the intercept alters the meaning of the logistic regression coefficients. So, best to keep it even if it is not significant.

The main issue here was the data treatment I was doing prior to training the algorithm related to balancing classes in my data: I using up sampling. This violates the independence across samples assumption. With other subset of data I had nearly separable classes, something that logistic regression can't deal with. So, either a modified version of the algorithm or another algorithm that handles this should be used.

In both cases this results in rather strange coefficients.

After correcting for this and considering the exponential ( coefficient) as odds ratio. I could find the expected interpretation by assuming 1-exp(coef) as the increase in odds of obtaining class 1 when the variable is increased by one unit.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.