1
$\begingroup$

I'm learning how to do logistic regression and I have some questions about verifying the model based on output in R. First below you can see the results from AIC analysis that chose the model with just Storage as the predictor to be the most appropriate. enter image description here

Next We have the model summary of this candidate model.

enter image description here

So my question here is, I'm looking at the p-value for the intercept and it looks like it is not significant. Is it important for the intercept to be significant given that often it has no meaning in the context of the problem? I.e. is this model invalidated by the insignificant y-intercept? Also, because it is logistic regression I think I have vaguely read that these p-values are not really used in logistic regression. So how about the scenario in which a simple linear regression has an insignificant y-intercept? Is that important?

Also look at the Residual Deviance. It is over dispersive I believe, at 5.173> 4 df. Now what should I do? Disregard the model, even though AIC said this is the best one?

Next is anova

enter image description here

Is there anything i should take away from this read out? Thanks.

$\endgroup$

1 Answer 1

3
$\begingroup$

$P$-values in GLMs

Using $p$-values for GLMs isn't a problem per se, as long as your model assumptions hold.$^*$ However, you have first selected the 'best' model based on AICc, so the $p$-value of Storage in the resulting model has lost its meaning (you already chose this model based on significance of this predictor, so the $p$-value is biased). You would be better off reporting the $p$-value of Storage in the original model, including all other predictors. In fact, if your goal is to report on the significance/effect sizes of these predictors, then there is no need for model selection, you could already do that with your full model.

$^*$: And as with any significance test, note that it is always better to include effect sizes rather than reporting $p$-values alone.

Significance of the intercept

This only tells you whether the linear part of the GLM crosses the y-axis significantly far away from $y = 0$.$^{**}$ Unless this is somehow important to your research question, you can usually ignore this test altogether. It certainly does not invalidate your model somehow.

$^{**}$: Or more correctly, $\eta = 0$, the linear part of the GLM.

Overdispersion

It is good that you consider the size of residual deviance ($5.173$) compared to the residual d.f. ($4$) However, I would say that these are actually quite close to each other and the $\chi^2$-statistic for overdispersion would be very low. As a rule of thumb, consider whether these two numbers are more or less in the same order of magnitude. If one is twice, or tens times as large as the other, your $p$-values are more reliable using a quasi-binomial distribution.


Significance in general

Most important, I think you should focus less on significance. Significance of the intercept is not important, insignificance is no reason for model selection and significance of the predictors by itself isn't very meaningful to begin with.

I recommend having a look at some of the Q&As here on stepwise regression (selecting variables based on significance) and why this is almost always a bad idea. A good place to start is here and here. You might have been taught in a course to select significant predictors, but this is not a good idea. Just to give you an idea of what to expect, here is a quote from the second linked post:

[A]ll predictors in a model and their posited causal relationship between a single exposure of interest and a single outcome of interest should be specified apriori. [...] Some journals (and the trend is catching on) will summarily reject any article which uses stepwise regression to identify a final model (Babyak, 2004), and I think the problem is seen in similar ways here.

$\endgroup$
2
  • 1
    $\begingroup$ That's is a comprehensive response @FransRodenburg, I don't have time to read it all now, but let me read it over the weekend and respond if I have some further questions before I accept the answer. Thanks. $\endgroup$
    – Bucephalus
    Commented Aug 8, 2018 at 20:51
  • $\begingroup$ Thankyou @FransRodenburg that was very helpful and informative. $\endgroup$
    – Bucephalus
    Commented Aug 10, 2018 at 12:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.