# When I remove a statistically insignificant dummy variable, shouldn't I remove all observations according to that variable?

I'm implementing a linear regression (OLS) on real estate data. I have a lot of dummy variables, where its value 1 indicate the presence of a characteristic, and 0 otherwise. Iteratively, I run the OLS, then check for the biggest statistically insignificant variable, that ones wich P > t are beyond 0.05. I take the greates P > t and remove it from the model.

But, now I realized I haven't removed the observations where the variables dumped in the process above was 1. Let's take as an example the number of rooms. In my model, this data goes from 1 to 4. So, I have three dummy variables, indicating if it has 2, 3 or 4 rooms. Let's say "2 rooms" is not significant, so I remove it from the model. I was letting all the "2 room" observations in the new model, but then I think they are being taken as the "1 room" variable, because the remaining "3 rooms" and "4 rooms" variables are 0.

Shouldn't I remove the observations according to the dummy variable removed?

• If the variable is insignificant, why should you remove a group of observations based on an insignificant value? Commented May 30, 2015 at 18:47
• @AlexR Because in the case of dummy variables, we are to use $k-1$ from $k$ classes of the original variable. In my example above, the "2 rooms" dummy is insignificant, then a remove this variable. My model now has "3 rooms" and "4 rooms" dummies, with others. The observations that has value 1 in the "2 rooms" variable are still in the dataset, but as their "3 rooms" and "4 rooms" columns are 0, I think regression will take these "2 rooms" observations as "1 room", misleading my interpretation in the end. This way of thinking is what is bugging me a lot. Commented May 30, 2015 at 18:53
• So you actually have a problem with the way you modelled the data. Time to make a "1 room" dummy variable. Then your old 2-rooms will just have $(0,0,0)$ in the three remaining dummies. Commented May 30, 2015 at 18:56
• It is usually nonsensical to test individual components of a dummy variable. Either all components should be included in the model or not. It may be more important to be aware that the approach of automatically removing large-p variables from the model is a poor strategy. Extensive discussion of the problems with it and descriptions of better strategies can be found by searching our site for such terms as "model fitting," "AIC," and "stepwise." It may also be worthwhile studying some of Frank Harrell's posts (stats.stackexchange.com/users/4253): he has written a well-received book about this.
– whuber
Commented May 30, 2015 at 19:02