0
$\begingroup$

My question is about removing non-significant predictors from a 'full' model to then evaluate a reduced model.

For clarification, I'm aware that this question has been asked before, but it has often been in the framework of model selection, and being concerned with finding an 'optimal' model. I'm more interested in removing terms when you have a clear model with a clear question.

Say I have a simple dataset with a continuous response variable (length), a continuous predictor variable (quality), and a categorical predictor variable (Site).

length       quality  Site
1  0.0793  0.2397161989 Site1
2  0.1725  0.0385483578 Site1
3  0.0581  1.9028360052 Site1
4  0.0525 -0.0925657836 Site1
5  0.1585 -0.8633296795 Site1
6  0.0817 -0.4840043812 Site1
7  0.1304 -0.7457158914 Site1
8  0.0572  0.3016842134 Site1
9  0.0333  1.1093523726 Site1
10 0.0584 -0.5282651197 Site1
11 0.0604  1.1132104925 Site1
12 0.0722 -0.3121098530 Site1
13 0.0574 -0.9638470029 Site1
14 0.0306 -0.6730752139 Site2
15 0.0605 -0.5219930835 Site2
16 0.0661  0.1020857212 Site2
17 0.2345 -0.7386738895 Site2
18 0.2243 -0.6268823297 Site2
19 0.0383  0.4200355418 Site2
20 0.0354 -0.0008132849 Site2
21 0.0874  0.7023074138 Site2

I want to know whether length is significantly related to quality, and whether it differs between site. To do this I fit a simple two-way model with an interaction

fit1<-lm(length~quality*Site, data=dat)
anova(fit1)

Response: length
             Df   Sum Sq   Mean Sq F value  Pr(>F)  
quality       1 0.011141 0.0111412  3.3305 0.08563 .
Site          1 0.000303 0.0003034  0.0907 0.76693  
quality:Site  1 0.004437 0.0044366  1.3262 0.26541  
Residuals    17 0.056869 0.0033452      

So in this example, the interaction term is non-significant. Should I then remove it and evaluate a reduced model? In this example, the reduced model is qualitatively identical so it's somewhat a moot point, but that aside, what is the correct procedure?

I've always thought you shouldn't remove it in an example like this. You have a clear question, fit a model to evaluate it. However other people in my lab think differently.

Is there a procedure here?

$\endgroup$
2
$\begingroup$

Even if it doesn't look like it, this is stepwise model selection, subject to all the usual criticisms. You're starting with one model and then changing to another on the basis of a $p$-value for a coefficient. There are many reasonable ways to do model selection; this isn't one of them.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.