0
$\begingroup$

I hope someone can help me understand why this is happening. Reproducible example in R:

var1 <- c(rep("a", 52), rep("b", 48))
var1 <- factor(var1, levels = c("b", "a"))
var2 <- c(rep("c", 47), rep("d", 53))
var2 <- factor(var2, levels = c("d", "c"))
outcome <- c(rep(1, 14), rep(0, 86))

testdf <- data.frame(var1, var2, outcome)
rm(var1, var2, outcome)

testmodel <- glm(outcome ~ ., data = testdf, family = "binomial")
summary(testmodel)

table(testdf$var1, testdf$outcome)
table(testdf$var2, testdf$outcome)

With regards to the outcome variable, var1a and var2c are both almost perfect predictors, both pointing to the same direction, i.e outcome = 1. Why do their coefficients have opposing signs, i.e. the coefficient for var1a is -1.761e-08, whereas the coefficient for var2c is 1.971e+01? This would suggest that being in var1a as opposed to var1b is associated with a decrease in the odds of the outcome but being in var2c as opposed to var2d raises the odds. But var1a and var2c are essentially the same group of observations as regards the outcome variable. How can one raise the odds and the other one decrease the odds?

This is happening in my real-life data and I need to understand the reason. I hope someone here can help by explaining. Thank you.

$\endgroup$

2 Answers 2

3
$\begingroup$

When you have more than one independent variable in a regression equation, the coefficient for each is after controlling for the other. In your example Var 1 and var 2 are very strongly related to each other. So, after controlling for var 1, there is little left for var 2 to explain and that part goes in the other direction.

Since all the variables are dichotomies, you can see this clearly in a table.

table(outcome, var1,var2)

Producing

, , var2 = d

       var1
outcome  b  a
      0 48  5
      1  0  0

, , var2 = c

       var1
outcome  b  a
      0  0 33
      1  0 14

You also have a lot of 0 cells.

Since this is not your real data (I hope your real data is less problematic), I suggest looking up "collinearity" and "controlling" either here or in a book on regression.

By the way, this is an issue in all forms of regression, not just logistic.

$\endgroup$
1
  • 1
    $\begingroup$ Thank you very much, Peter, I appreciate that you took the time to explain this. It does make sense now that you have pointed it out. $\endgroup$
    – Reader 123
    Commented Apr 30 at 15:01
1
$\begingroup$

Illustrating Peter Flom's answer, once we know whether we're on the top or bottom of this plot (var2), whether we're on the left or right (var1) tells us nothing we didn't already know.

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.