# Help interpreting logistic regression

I'm new to R and logistic regression and have to admit that I don't really know how to interpret the result. I'm trying to compute a pretty simple model with 2 predictors (A and B). When I first try to compute models with the predictors one by one they are both significant. When I put them together and add an interaction term they lose their significance (but the interaction term is weakly significant). I interpret this as A and B are overlapping and no longer significant when the oter parameter is hold constant. Right?

But now to the part I don't know how to interpret. I make predictions from my models (see code below) and then run t-tests for the predictions vs. the depending variable. I think this should give a hint on how good the model is (is there a better way?). When I do it this way I get a much lower p-value for the model with both A and B. I think this is contradictory. The first part tells me that A doesn't provide any significant information to the model when combined with B, but on the other hand I get much better predictions. I guess something is really wrong, but I can't figure out what. Can you help me?

model1=glm(f~A, , family=binomial(link="logit"))
summary(model1)
summary(model2)
summary(model3)
p1=predict(model1, newdata=data, type="response", na.rm=TRUE)
p2=predict(model2, newdata=data, type="response", na.rm=TRUE)
p3=predict(model3, newdata=data, type="response", na.rm=TRUE)
t.test(p1~f)
t.test(p2~f)
t.test(p3~f)


Part of the output:

> summary(model1)
Estimate Std. Error z value Pr(>|z|)
(Intercept)  -1.9756     0.3499  -5.647 1.64e-08 ***
A            -0.5898     0.2119  -2.784  0.00537 **

> summary(model2)
Estimate Std. Error z value Pr(>|z|)
(Intercept)  8.354e-01  1.309e+00   0.638   0.5234
B           -1.028e-04  5.122e-05  -2.007   0.0447 *

> summary(model3)
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept)  1.254e+00  1.705e+00   0.735    0.462
A            1.589e+00  9.743e-01   1.631    0.103
B           -1.324e-04  7.333e-05  -1.805    0.071 .
A:B         -9.418e-05  4.632e-05  -2.033    0.042 *

> t.test(p1~f)
t = -2.614, df = 11.83, p-value = 0.02286

> t.test(p2~f)
t = -1.8702, df = 15.679, p-value = 0.08024

> t.test(p3~f)
t = -4.9777, df = 17.344, p-value = 0.0001084

• Are you certain that you want to use A*B and not A+B? Using the t-test is interesting. Have you also calculated precision and recall? – John Yetter Mar 21 '14 at 17:37
• Note that model1 has a typo. There is either an extra comma , or a missing argument (but then the argument is presumably missing in the other models as well). – gung - Reinstate Monica Mar 21 '14 at 17:40

There are several issues here:

1. As @John Yetter notes, you have included the interaction A*B in model3 as well as the two variables A & B. If you only wanted to fit a model with the two variables, you would use: glm(f~A+B, family=binomial(link="logit")).

If you do want the interaction / when you have one included in the model, the interpretation of the main effects differs. Specifically, the main effect of A is the relationship between A and f when B is equal to 0 (and vice versa for B).

2. The relationship between regressing y on x, and regressing x on y is, in general, not the same. This is especially true for logistic regression (where there is a non-linear transformation in between y and x, but not between x and y) and for multiple regression (logistic or otherwise). That is, you cannot turn around your first model f~A into a t-test A~f.

3. You should not test the relationship between the model's predicted values and the original response value as a way to determine "how good the model is". This makes no sense. Since the predicted values are a perfect (i.e., non-stochastic) function of the predictor variables, there is no new / different information there. A more typical way to understand the quality of the model is to examine the ROC curve.

• Thanks for answering! 1) I tried with and without the interaction term and the reason for choosing to include it is a better outcome (sensitivity/specificity). – user42346 Mar 31 '14 at 19:30
• Thanks for answering! 2) I don't understand this part. When I try to turn the t-test around (f~A) I get the error message "grouping factor must have exactly 2 levels". I agree that it looks weird when they aren't in the same order, but that's the only way to avoid error messages. How should it be? – user42346 Mar 31 '14 at 20:33