What to do with dummy variable that is not significant? [duplicate]

I´m running a binary logistic regression to predict the purchase probability for a product. My model contains mostly dichotomous and categorical variables. One variable, let´s say "decision maker", has three categories "x1 = CEO, x2= purchasing manager, x3= others".

So, what to do when the variable "decision maker" is significant but the dummy variable "x2= purchasing manager" is not significant?

• Welcome on SE. Can you detail a bit more? It's hard for me to understand your problem. – mic Dec 22 '15 at 11:03
• You should keep it in your model, because otherwise people who fall into that category will just go to the reference category, along with people who were in the original reference category, which makes your interpretation trickier. – Marquis de Carabas Dec 22 '15 at 22:44

Allow me to provide a quick example. Suppose you have data $\{x_i,y_i\}_{i=1}^n$ where $x_i$ is trichotomous, taking possible values $a, b$, or $c$. Then, we may consider the regression $$\mathbb{E}\left[ y_i\mid x_i \right] = \alpha + \delta_a I(x_i = a) + \delta_b I(x_i = b) + \delta_c I(x_i = c)$$ where $I(\cdot)$ is the indicator function and the $\delta$ are regression coefficients. (Note that $\alpha$ is not necessary due to the contrasts argument made previously; we will keep it just so that this is intuitive.) Now, if you find that you fail to reject the null hypothesis $H_0: \delta_c = 0$, then this would imply that we cannot refute the claim that $\mathbb{E}[y_i \mid x_i = c] = \alpha$.
Dropping the insignificant regressors is the variable-selection procedure inherited from classical statistics and was more practical perhaps in the days of small data. As @björn suggests below, you may also consider using other selection or regularization procedures. For example, if you care for prediction, you may consider including a ridge penalty or if you care for sparsity, consider the LASSO or newer sparsity-inducing penalties for linear regression. Both of these regularization methods are fast and easy to tune via cross-validation. You may also consider AIC, BIC, and Mallow's $C_p$ selection criterion as an alternative to the stage-wise approach I suggested.