I have 4 predictors, and 1 binary response. I fitted a logistic regression model. A strange thing is that all the coefficient of the model are negative. Is that possible? Probably I did something wrong. My interpretation is that the odds ratio of either variable is less than 1. That is, neither variable actually do any good to the response. Even the intercept is negative. Please share your intelligence.
$\begingroup$
$\endgroup$
1
asked Mar 1, 2013 at 21:06
-
3$\begingroup$ For some perspective: If the model explains nothing, so that the coefficients are essentially randomly distributed around zero, then there is a one in eight chance that all four coefficients have the same sign (one in 16, including the intercept). Such chances are not in the least unusual. $\endgroup$– whuber ♦Commented Mar 1, 2013 at 21:30
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
Yes, it is possible. Couple of things here. The direction of your predictors are critical to the interpretation; if they were scaled in the opposite direction, they would be positive. Second, it would seem on the face that your predictors lower the log odds given a unit change, which in itself might be good if say the outcome is death. Also, you might consider centering some of your predictors if the interecpt does not seem interpretable.
$\begingroup$
$\endgroup$
From your description I see nothing out of the ordinary.
That the intercept is negative corresponds to that the estimated probability of the response is less than 50% when all model covariates equal zero.
If the coefficients of the model covariates are negative, then yes, the corresponding odds ratios are smaller than 1. If this is unexpected given your data, you may need to check how your covariates are coded.
$\begingroup$
$\endgroup$
5
I didn't really get logistic regression until I thought about it this way:
- Picture the S curve (the logistic) going from -3 to 3
- Look at the coefficient estimate on your constant term. Mark it on the x axis of the S curve.
- Each coefficient moves you $\beta$ units along the X axis of the S curve. If you want to know what probability that corresponds to, go up from the X axis to the S curve and then over to the Y axis.
So, say that your intercept is, like, -.5. This is something like 40% probability (or so). Say your first beta is -.2 or something. This means that you follow the X axis over to -.7, which has a lower probability. Say you have a coefficient that is -5. That'll take you way out left, where chances are basically zero.
Its really pretty simple when you break it down simply.
answered Mar 2, 2013 at 12:51
-
1$\begingroup$ Could you clarify how these thoughts address the question in this thread? And what is your justification for concluding the intercept might be larger than $-2$? After all, if all the independent values were negated, then (1) the intercept would not change but (2) all coefficients would become positive. Moreover, if the units of measurement of the IVs were changed, then their coefficients would change in proportion; and if the IVs were recentered, the intercept could change to almost anything. Thus it's difficult to see how you can conclude anything at all about the coefficients. $\endgroup$– whuber ♦Commented Mar 2, 2013 at 21:02
-
$\begingroup$ I'm not definitively concluding anything. All that I'm saying is that -2 on the logistic curve corresponds to a pretty low "baseline" probability, speaking loosely. If all of the coefficients are negative, then upward movement in the variables that they represent will take the probability of the outcome lower. I suppose that it is possible that the poster is posting about a dataset with extremely low probabilities, or variables that might more intuitively be rescaled by -1. But I was just typing to help provide a bit of intuition to augment the more formal answers above. $\endgroup$ Commented Mar 3, 2013 at 17:47
-
$\begingroup$ Think about what value the intercept would have to have if, say, one of the IVs were years since 1900 and the data were from a short experiment in 2013 during which probabilities decreased by a factor of 2 on average. This illustrates one of the points I was trying to make: without explicit assumptions about the variables, you cannot assert anything about the intercept, not even anything "intuitive" or qualitative. $\endgroup$– whuber ♦Commented Mar 3, 2013 at 18:28
-
$\begingroup$ You're not wrong, and I've deleted the offending line. Still, I think that my answer is a useful way to convey intuition to novices. $\endgroup$ Commented Mar 4, 2013 at 2:57
-
1$\begingroup$ Perhaps it does convey intuition: after all, if it works to help you understand the situation, it will likely help others. Maybe posting an illustration of what you're doing would make your idea clearer. $\endgroup$– whuber ♦Commented Mar 4, 2013 at 15:33