# GLM model validation

When performing model validation and we are dropping the least significant explanatory variables until we find the optimum model where all remaining variables are significant, how does one go about dropping a particular "unimportant level" within a nominal variable? In my output below I have run a Neg. bin. GLM inorder to test abundance against the various explanatory variables listed. (as.factor(JJ)2) is the most obviously insignificant variable that I would like to drop but because it is a factor with three levels I don't know how to do this. (Of course I will have to rejig the data to unmask baseline values that are within the intercept also) Thanks!

Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept)     3.351414   0.945861   3.543 0.000395 ***
DD              0.039417   0.011373   3.466 0.000528 ***
EE             -0.001497   0.001015  -1.475 0.140093
PP             -0.138303   0.094287  -1.467 0.142423
as.factor(BB)2 -0.901877   0.365796  -2.466 0.013682 *
as.factor(JJ)1  0.495985   0.198394   2.500 0.012420 *
as.factor(JJ)2 -0.276830   0.374576  -0.739 0.459878
as.factor(VV)2 -0.499569   0.174710  -2.859 0.004244 **
as.factor(VV)3 -0.166065   0.175430  -0.947 0.343834
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for Negative Binomial(4.8369) family taken to be 1)

Null deviance: 97.392  on 46  degrees of freedom
Residual deviance: 47.883  on 38  degrees of freedom
AIC: 401.37

-
What's your model? – Roman Luštrik Aug 18 '11 at 11:25
What formula did you use for constructing this model? – Roman Luštrik Aug 18 '11 at 11:32
@Platypezid As you seem to have been performed stepwise regression, please update your question with the precise R commands you used and the structure of your data (I can see three numerical variables and three categorical variables with 2 or 3 levels). – chl Aug 18 '11 at 11:43
@Platypzeid - stepwise regression is almost always a bad idea, even when you do it by hand. – richiemorrisroe Aug 18 '11 at 12:26
Stepwise regression destroys all aspects of statistical inference (bias, type I error, false standard errors, inflated regression coefficients, inflated $R^2$, false confidence interval coverage). Breaking apart components of categorical variables is even worse. – Frank Harrell Aug 18 '11 at 14:31

You can check the significance of a nominal variable without inspecting individual level effects by using the likelihood ratio test, comparing the model with and without the variable. I've answered this exact question in Checking if a nominal variable is important in a GLM model

-
I understand the loglik() approach, however, what I really want to know is whether you can keep one level of a nominal variable in a model to help better explain the data. i.e. If significantly more birds were found in relation to a particular level of say a tree canopy with three levels (e.g. low in comparison to medium and high) then isn't this important in the optimum model? – Platypezid Aug 18 '11 at 13:29
Checking individual levels is, in some sense, a "post-hoc" test that is usually done after determining (via some omnibus test) that the variable as a whole is significant. So, I would advise testing the variable and, if significant, consider pairing down to sub-levels (although that may cause interpretability problems). It might be best to go "all or nothing" when doing model selection. – Macro Aug 18 '11 at 13:48
Well put, @Macro – Frank Harrell Aug 18 '11 at 14:29
@Macro Yes, thanks very much. This is a valid take on the problem and explained with a deep understanding, thanks a million, I see the issues now. – Platypezid Aug 18 '11 at 14:38
@Platypezid - Glad I could help. Also, just in general, if you have found an answer helpful and definitive, please consider accepting the answer - I noticed you've yet to accept an answer on the several questions you've asked. – Macro Aug 18 '11 at 14:58

In general, I am wary of step-wise variable elimination and model selection. I would suggest using the lasso, elastic net, or ridge regression for variable penalization and selection. Check out the R packages lars, glmnet, and lm.ridge in MASS.

-
Thanks a lot for this additional comment – Platypezid Jan 25 '12 at 15:22