# What to do with dummy variable that is not significant? [duplicate]

I´m running a binary logistic regression to predict the purchase probability for a product. My model contains mostly dichotomous and categorical variables. One variable, let´s say "decision maker", has three categories "x1 = CEO, x2= purchasing manager, x3= others".

So, what to do when the variable "decision maker" is significant but the dummy variable "x2= purchasing manager" is not significant?

• Welcome on SE. Can you detail a bit more? It's hard for me to understand your problem.
– mic
Dec 22, 2015 at 11:03
• You should keep it in your model, because otherwise people who fall into that category will just go to the reference category, along with people who were in the original reference category, which makes your interpretation trickier. Dec 22, 2015 at 22:44

Dummy variables are about contrasts. In other words, the significance of a dummy (unlike a quantitative covariate) is not necessarily if it is significantly different from zero (though it can be), but rather that there is a contrast between the positive and negative classes. It seems you have a categorical variable where one of the categories does not contrast with the others. The proper variable selection mechanism would be to allow the observations in this category to be explained only by the intercept.

Allow me to provide a quick example. Suppose you have data $\{x_i,y_i\}_{i=1}^n$ where $x_i$ is trichotomous, taking possible values $a, b$, or $c$. Then, we may consider the regression $$\mathbb{E}\left[ y_i\mid x_i \right] = \alpha + \delta_a I(x_i = a) + \delta_b I(x_i = b) + \delta_c I(x_i = c)$$ where $I(\cdot)$ is the indicator function and the $\delta$ are regression coefficients. (Note that $\alpha$ is not necessary due to the contrasts argument made previously; we will keep it just so that this is intuitive.) Now, if you find that you fail to reject the null hypothesis $H_0: \delta_c = 0$, then this would imply that we cannot refute the claim that $\mathbb{E}[y_i \mid x_i = c] = \alpha$.

The encoding of this sort of absence of a category, if you will, will be observed in the design matrix used for this regression. For more on this topic, consider the literature on the matrix algebra parametrization of linear regression.

Dropping the insignificant regressors is the variable-selection procedure inherited from classical statistics and was more practical perhaps in the days of small data. As @björn suggests below, you may also consider using other selection or regularization procedures. For example, if you care for prediction, you may consider including a ridge penalty or if you care for sparsity, consider the LASSO or newer sparsity-inducing penalties for linear regression. Both of these regularization methods are fast and easy to tune via cross-validation. You may also consider AIC, BIC, and Mallow's $C_p$ selection criterion as an alternative to the stage-wise approach I suggested.

• Just because something is not statistically significant surely does not mean you should drop it from the model. Certainly not, if you are interested in out-of-sample predictions. Dec 22, 2015 at 12:21

There is no answer because the question "what to do?" is too vague. If your research question is about there being heterogeneity across the three categories then you report that result and happily confirm your hypothesis. If your question was about that specific level being different from the reference group, then it looks like your hypothesis didn't pan out.