3
$\begingroup$

I'm running a logistic regression model where anecdotally I expected age to be a very large factor. If you see from the charts I made in Excel before running the model through R, this is how the support lines up by age:

enter image description here

Looks pretty significant.

Though when I run the model, as you can see below, age is the only thing that's not significant -- which was very surprising:

> attach(mydata) 
> 
> # Define variables 
> 
> Y <- cbind(support)
> X <- cbind(sex, region, age, supportscore1, supportscore2, county)
>
> # Logit model coefficients 
> 
> logit <- glm(Y ~ X, family=binomial (link = "logit"), na.action = na.exclude) 
> 
> summary(logit) 

Call:
glm(formula = Y ~ X, family = binomial(link = "logit"), na.action = na.exclude)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-2.1019  -0.7609   0.5231   0.7101   2.3965  

Coefficients:
                       Estimate Std. Error z value Pr(>|z|)    
(Intercept)            4.013446   0.440962   9.102  < 2e-16 ***
Xsex                  -0.229256   0.104859  -2.186 0.028792 *  
Xregion               -1.103308   0.091497 -12.058  < 2e-16 ***
Xage                   0.004569   0.003209   1.424 0.154512    
Xsupportscore1        -0.019262   0.005732  -3.360 0.000778 ***
Xsupportscore2         0.019810   0.005264   3.764 0.000168 ***
Xcounty               -0.047581   0.011161  -4.263 2.02e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 2871.5  on 2072  degrees of freedom
Residual deviance: 2245.5  on 2066  degrees of freedom
  (66 observations deleted due to missingness)
AIC: 2259.5

Number of Fisher Scoring iterations: 4

My only guess on this is that the previous support scores (both 0-100 numerical values) I'm using may have already taken age into account, and the model doesn't want to count it twice. Though, to compare, region and county are just two different ways of cutting up the geography -- and those both seem significant.

Can somebody let me know what you would think if your model told you that age wasn't significant when in clearly is? Trying to figure out if there's a way of thinking about it that I'm missing or if something in my code is wrong.

Thanks!

-- EDIT

Pairs plot added to show correlation (despite some factors being categorical):

pairs(~sex + region +  age + supportscore1 + supportscore2 + county, data=mydata)

enter image description here

Improve this question
$\endgroup$
7
  • 4
    $\begingroup$ Simpson's paradox is addressed in dozens of questions on site (both with and without using that specific term). If you leave out an important variable, you may see an effect that disappears after you adjust for the previously omitted one. [There's also the possibility that it's important but that multicollinearity in predictors has inflated standard errors. It can also happen when sample sizes are small - not an issue here, I think. There are so many ways what you see can happen but my guess is another variable has captured it.] $\endgroup$ – Glen_b May 1 '14 at 1:08
  • 3
    $\begingroup$ This is all nothing to do with R, by the way. It would be the same in any other package. $\endgroup$ – Glen_b May 1 '14 at 1:14
  • 1
    $\begingroup$ This is a FAQ. Here are some threads that can help you: How can adding a 2nd IV make the 1st IV significant?, Estimating b1x1+b2x2 instead of b1x1+b2x2+b3x3, How can a regression be significant yet all predictors be non-significant?, & Significant t-test vs non-significant F statistic. You might also read some of the threads on the site categorized under the simpsons-paradox tag. $\endgroup$ – gung - Reinstate Monica May 1 '14 at 1:25
  • 1
    $\begingroup$ You're welcome, @Ryan. Your question is clear & well-formed. It's just that it has been asked & answered (in different terms / contexts) many times on CV. Read some of the linked threads. If you still have a question afterwards, come back here & edit your question to state what you've learned & what you still need to know, & then we can provide a non-duplicate answer. Here's another: Basic Simpson's paradox. $\endgroup$ – gung - Reinstate Monica May 1 '14 at 1:38
  • 1
    $\begingroup$ @gung Definitely -- I don't mind it being marked as a duplicate. I would've likely found it first if I knew it was called Simpson's paradox. Now I do! Thanks again. $\endgroup$ – Ryan May 1 '14 at 1:46