# Feature Engineering : combine a categorical Feature and a continuous Feature

When we analyze data, we can observe several variables that may contain mutual information. For an example, there can be a binary variable such as $$Y=\text{Have you ever smoked?}$$. Then there will be a follow up question such as (in this case it is a continuous variable) "How old were you when you first smoked?"

For the variable X that measures "How old were you when you first smoked?",

\begin{align} X_1 &= \begin{cases} 0 & \text{if never smoked} \\ 1 & \text{if smoked} \end{cases} \\ X_2 &= \begin{cases} 0 & \text{if } x_1 = 0 \geq 0 & \text{if } x_1 = 1 \end{cases} \end{align}

So the distribution of $$X_2$$ will be like this:

That means it contains several zeros, as it is depends on the previous question ($$X_1$$).

One way to deal with this type of problem is calculate the age of first smoke ($$X_2$$) only for users (i.e., eliminating zeros). Then the drawback is that it will reduce the sample size with respect to $$X_2$$ variable.

Another way to model $$X_2$$ is convert it to a categorical variable. For an example someone can do like this:

$$$$X_2^{\text{categorized}} = \begin{cases} \text{Never Smoke} & X_2 = 0 \\ \text{Young} & 0 < X_2 \leq 15 \\ \text{Middle} & 15< X_2 \leq 20 \\ \text{Old} & 20 < X_2 \end{cases}$$$$

But is there way to model $$X$$ by preserving the continuous nature using a mixture distribution? I mean mixture distribution in the sense that, this may be something like the product of $$X_2$$ and $$X_1$$. However, I am not sure how to do this.

Since in this case $$X_1$$ is binary, taking the product of $$X_2$$ and $$X_1$$ seems to make sense. But I am not sure how this will work in general, i.e., when $$X_1$$ has more than 2 categories.

• Why do you need this combined into a single variable? It seems to me that if you did, you'd need to account for the fact that the relationship might differ for the never smokers, which would get you back to having multiple variables. Commented Mar 16, 2020 at 0:44
• @gung-ReinstateMonica I may need to combine because there can be situations where the number of zeros in X are significantly higher. In such situations, my opinion is dealing only with X can be misleading. Commented Mar 16, 2020 at 0:52
• @gung-ReinstateMonica In this case also there about 150 cases of zeros for X variable Commented Mar 16, 2020 at 0:53
• I'm not sure I see the problem with that. Are these the response variable[s], or are these predictor variables? Commented Mar 16, 2020 at 4:26
• @gung-ReinstateMonica All are predictors . I changed the notation in the question. I apologize for any confusion. Commented Mar 16, 2020 at 13:55

This is probably just a hack that does not solve this kind of problem in general, but may be well-suited for your problem: a person that does not smoke is equivalent to a person that starts smoking at the age of infinity. Hence if you transform your $$X2$$ into $$X2' = 1/X2$$, then a person that never smoked should have a value $$0 = 1 / \infty$$, while other people just have $$1/X2$$. If you are doing some kind of linear regression, this will destroy the original linearity, but should be fine for nonlinear regression techniques.