I was trying to get an intuition for the interpretation of the coefficients in a logistic regression that was intended to reproduce to some extent that presented in a youtube video (http://youtu.be/vq-_4kWmzTo). So I created a fictitious data set reflecting the chances of getting accepted (Accepted: int: 0 0 ... 1 0) into a college as related to SAT scores (int 1136 1347 1504) and family/ethnic background (categories = "red" vs "blue").

fit <- glm(Accepted ~ Background - 1,data=dat, family="binomial")
exp(cbind(OR = coef(fit),confint(fit)))


                      OR     2.5 %    97.5 %
Backgroundblue 0.7088608 0.5553459 0.9017961
Backgroundred  1.7352941 1.3632702 2.2206569

The interpretation seemed easy: Red applicants have 1.7 times more chances of getting in over the rest; blue applicants were at a disadvantage, and had 7 over 10 odds of getting in.

However, the more complete model,

fit <- glm(Accepted ~ SAT.scores + Background - 1,data=dat, family="binomial")
exp(cbind(OR = coef(fit),confint(fit)))

yielded coefficients for background that are difficult to reconcile or interpret:

                         OR        2.5 %       97.5 %
SAT.scores     1.008558e+00 1.006940e+00 1.010297e+00
Backgroundblue 8.730056e-06 8.459031e-07 7.634723e-05
Backgroundred  2.329513e-05 2.426748e-06 1.929259e-04

Can you help point out what I am missing? Thank you.

Thanks to the enlightening answer from Maarten below, I was able to make some progress, and obtain the correct Odds Ratios without and with the SAT confounder:

Here is just regressing to Background ("Red" versus "Blue"):

fit <- glm(Accepted ~ Background, data = dat, family = "binomial")
exp(cbind(Odds_Ratio_RedvBlue = coef(fit), confint(fit)))

                        Odds_Ratio_RedvBlue             2.5 %       97.5 %
(Intercept)             0.7088608                     0.5553459   0.9017961
Backgroundred           2.4480042                     1.7397640   3.4595454

Which brought up a couple of additional questions (probably very basic): 1. Why is the Odds Ratio of the (Intercept) - 0.7088608 - the same for this model as for the Odds - as opposed to Odds Ratio - of the Background Blue in the model without the intercept above? And 2. Shouldn't the OddsRatio of Blue and Red be the reciprocal of each other $OddsRatioBlue = 1 / OddsRatioRed$?

  • $\begingroup$ Should you not avoid '-1' so that intercept is also included? Are the ORs more realistic without '-1' ? $\endgroup$
    – rnso
    Feb 3 '15 at 11:59

Your interpretation of the numbers in your model is incorrect because you used an unusual (but not incorrect) parameterization. You omitted the constant and included both categories of family background. As a consequence the coefficients labeled OR are not Odds Ratios but just odds. In this case the odds of being accepted for someone with blue background in your first model is 0.71, which you can interpret as for every person with blue background that is not accepted you expect to find 0,71 persons with blue back ground that is accepted. Notice that this does not involve a comparison between blue and red backgrounds, it just a quantification of the likelihood that someone with blue background is accepted. If you had included the constant and excluded the variable backgroundred, your interpretation would be correct (but the numbers would obviously be different). For more on these different but equivalent parameterizations see here.

By adding the (uncentered) SAT scores your background variables now estimate the odds of being accepted for someone with an SAT score of 0. This odds is not surprisingly very very small regardless of family background. You can get more meaningful estimates when you center SAT scores at some meaningful number. Say that meaningful number is 1200 (I know next to nothing about SAT scores), than you just create a new variable that is the original SAT scores minus 1200 and you add that new variable to your model instead of the original. Your background variables now measure the odds of being accepted for someone with an SAT score of 1200.

Additional question 1: your original model and your new model are just different ways of saying the same thing, so it is not a surprise that some coefficients are the same. As you can guess, the exponentiated constant is not an odds ratio but an odds. To be precise the odds of success for the reference category, in your case blue background. Your old model said "This is the odds of success for blue background, and that is the odds of success for the red background". Your new model says "This is the odds of success for the blue background, and the odds of success for someone with a red background is a factor such and so smaller/larger". For the two models to be equivalent the exponentiated constant in your new model should be the same as the exponentiated coefficient of backgroundblue in your old model. The ratio of exponentiated coefficients of backgroundblue and backgroundred in your old model should be the same as the exponentiated coefficient of backgroundred in your new model. This is indeed the case $\frac{1.7352941}{0.7088608}=2.448004$.

Additional quesiton 2: yes, try and estimate your new model with backgroundblue instead of backgroundred and you'll see that that is the case.

  • $\begingroup$ This is a superb explanation. $\endgroup$ Feb 3 '15 at 17:45
  • $\begingroup$ I edited my original post with a couple of follow-up questions at the end. $\endgroup$ Feb 3 '15 at 18:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.