Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I´m running a binary logistic regression to predict the purchase probability for a product. My model contains mostly dichotomous and categorical variables. One variable, let´s say "decision maker", has three categories "x1 = CEO, x2= purchasing manager, x3= others".
So, what to do when the variable "decision maker" is significant but the dummy variable "x2= purchasing manager" is not significant?
Dummy variables are about contrasts. In other words, the significance of a dummy (unlike a quantitative covariate) is not necessarily if it is significantly different from zero (though it can be), but rather that there is a contrast between the positive and negative classes. It seems you have a categorical variable where one of the categories does not contrast with the others. The proper variable selection mechanism would be to allow the observations in this category to be explained only by the intercept.
Allow me to provide a quick example. Suppose you have data $\{x_i,y_i\}_{i=1}^n$ where $x_i$ is trichotomous, taking possible values $a, b$, or $c$. Then, we may consider the regression $$\mathbb{E}\left[ y_i\mid x_i \right] = \alpha + \delta_a I(x_i = a) + \delta_b I(x_i = b) + \delta_c I(x_i = c)$$ where $I(\cdot)$ is the indicator function and the $\delta$ are regression coefficients. (Note that $\alpha$ is not necessary due to the contrasts argument made previously; we will keep it just so that this is intuitive.) Now, if you find that you fail to reject the null hypothesis $H_0: \delta_c = 0$, then this would imply that we cannot refute the claim that $\mathbb{E}[y_i \mid x_i = c] = \alpha$.
The encoding of this sort of absence of a category, if you will, will be observed in the design matrix used for this regression. For more on this topic, consider the literature on the matrix algebra parametrization of linear regression.
Dropping the insignificant regressors is the variable-selection procedure inherited from classical statistics and was more practical perhaps in the days of small data. As @björn suggests below, you may also consider using other selection or regularization procedures. For example, if you care for prediction, you may consider including a ridge penalty or if you care for sparsity, consider the LASSO or newer sparsity-inducing penalties for linear regression. Both of these regularization methods are fast and easy to tune via cross-validation. You may also consider AIC, BIC, and Mallow's $C_p$ selection criterion as an alternative to the stage-wise approach I suggested.
There is no answer because the question "what to do?" is too vague. If your research question is about there being heterogeneity across the three categories then you report that result and happily confirm your hypothesis. If your question was about that specific level being different from the reference group, then it looks like your hypothesis didn't pan out.
