What is the appropriate way to encode and handle regressors that are nested? For example, a feature "Husband's age" is nested in the feature "Is married" in a sense that it's not defined unless "Is married" = 1. Should "Husband's age" be set to 0 or NA for those who are not married? Should we omit "Husband's age" from the model entirely and only use the interaction term "Husband's age"*"Is married"? I guess this situation must be quite common, but I don't even know how to google it.
$\begingroup$
$\endgroup$
-
1$\begingroup$ See Machine learning feature encoding or How to handle with missing values in order to prepare data for feature selection with LASSO?. This case is not applicable rather than missing, though I'm not sure myself of the terminology. (Note that "setting to NA" doesn't mean much - unless you also explain how you're going to treat NA in your analysis.) $\endgroup$ – Scortchi - Reinstate Monica♦ Jun 2 '16 at 14:49
-
$\begingroup$ Great! "Not applicable" is definitely the case. Thank you! $\endgroup$ – Evgeniy Riabenko Jun 2 '16 at 20:26
$\begingroup$
$\endgroup$
You deal with these "conditional variables" by including the variable only as an interaction with the original variable, without including it as a base effect. That way it only enters into the model in the case when the underlying condition for the existence of this variable is met (see similar questions and answers here and here). So, if you have an indicator variable married with a nested continuous variable spouse_age then you would use a model form like this:
Response ~ 1 + ... + married + married:spouse_age
Notice that in this model form there is no main effect for spouse_age because this is a conditional variable that only make sense if the respondent has a spouse (i.e., if the married indicator variable is one). Also, you will notice that with this model form it does not matter how you code the variable spouse_age for a non-married respondent (although NA makes more sense in the data).
$\begingroup$
$\endgroup$
I think you do not need to do anything with it. Essentially you are asking how to deal with "correlated" features, i.e., two random variables $X_1$ and $X_2$ they are not independent where $P(X_1,X_2)!=P(X_1)P(X_2)$. This case is very normal in most real world problems.
For example, you want to predict a person's income, it is very common to use both age and education level as features, and age and education level are not independent, e.g., higher education usually indicates the person is older. Another example would be predicting a person's healthy status. It is very common to use both height and weight. Although we know height and weight has really strong correlation.
In your example, the only difference is you have discrete random variables and the two examples I mentioned above are continuous random variables. For your case, Let's use $X_1$ to represent spouse's age and $X_2$ for is married. You will have, say $P(X_1|X_2=1)$ is normal distribution, while $P(X_1|X_2=0)=0$.
As well as you do not have highly correlated features, such as identical features, e.g., $P(X_1)=P(X_2)$ / "weight in lb and weight in kg", you would not have too much problems. If you do have "collinearity problem", you can drop some "repetitive" features or use Principal component analysis or use regularization
-
$\begingroup$ Thanks, but correlation have nothing to do with the problem. I wouldn't want to exclude or transform these features, I just want to properly encode them. $\endgroup$ – Evgeniy Riabenko Jun 2 '16 at 20:34
