I have 5 single factor regression models. Each has intuitive (positive) coefficients which is good. If I run multiple logistic regression one of the coefficients turns negative.

If I run stepwise selection, the negative coefficient stays in.. it's not redundant..

How should I deal with this? Is there a way to enforce positive coefficients?


  • 4
    $\begingroup$ Isn't it possible that your independent variable that obtains negative coefficient in multiple-prediction circumstances serves as a suppressor variable? $\endgroup$ – ttnphns Jul 6 '11 at 10:57
  • 2
    $\begingroup$ then probably you have some multicollinearity in your regressands (there is simply too many of them you want to include), try principal components for instance for "bad habits" and "good habits" or other interpretable results. $\endgroup$ – Dmitrij Celov Jul 6 '11 at 12:41
  • 2
    $\begingroup$ @Dmitrij @user Multicollinearity is unlikely to be the problem. A change of sign when a new independent variable is introduced is a strong signal of a possible interaction. The next step--which should be done in any event--is to investigate the pairwise interactions. $\endgroup$ – whuber Jul 6 '11 at 15:29
  • 1
    $\begingroup$ If you want to restrict the coefficients to be positive, that's easy to accomplish, e.g., using the log-barrier method (or any off-the-shelf constrained optimization tool). The log-likelihood function for logistic regression is concave, so its restriction to a convex region (the positive orthant) is also concave, meaning there's a unique global maximum. But I concur with the other commenters: restricting them to be positive isn't necessarily the right thing to do. $\endgroup$ – jpillow Jul 6 '11 at 16:06
  • 2
    $\begingroup$ @user Please explain why that's a "disaster"! Remember, when interactions are present, the coefficients alone are (almost) meaningless: to assess the effective slopes, you need to evaluate the partial derivatives of the response with respect to variates at selected values of the variates. $\endgroup$ – whuber Jul 19 '11 at 21:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.