Handle missing value in continuous variable in Decision Tree without losing its meaning

Question

I want to train a Decision Tree model with a dataset, of which some of the continuous variables contain missing values.

I want to preserve the meaning of missing value while training, meaning that I want "NA" (or any value that represents "NA") in continuous variables to show up in the rules of a specific node (if it matters) in the trained model and not just to replace it with medium or other values.

Is there any suggestions as to how should I impute the missing values? I use rpart to build decision tree.

Thank you so much!

Answer 1

There are many strategies to impute values:

• mean imputation
• last value carried forward
• using information from related observations
• ...

If you are interested in "preserv(ing) the meaning of missing value", one idea could be to replace NAs with values far off from the domain of the available data in that specific feature.

As an example, we built an algorithm to analyze text snippets and extract information. As part of the features, we encoded the structure of sentences as distances (in number of words and characters) between tokens (special words) found in the text snippets. If the token was not present in the text, we would assign a very high value to the distance. Decision Trees will slice the domain space accordingly.

Another idea would be to impute the missing values to "likely" values (ex: mean imputation) for the feature F but generate an additional feature one-hot-encoding whether the data in F was imputed or not.