I have a dataset with both continuous, categorical, and ordinal variables (roughly 1k of them), over 100 Million rows, and a lot of missing values. I am aware of various methods for imputation, but I'm worried these will either be overly biased (mean or mode imputation, etc.) or unfeasible given the size of the data (MI, other modeling imputations).
I am planning on using LASSO (logistic regression with an L1 penalty) to predict a binary response variable.
I have an idea that I would love to hear people's thoughts on: just add a one-hot encoded variable for each variable with any missing values that would indicate a missing value for that variable.
For the categorical variables, I will be one-hot encoding them anyway, so this seems reasonable. For the continuous and ordinal variables, I would replace the missing values with zero, and then put a 1 in corresponding one-hot "missing" indicator variable.
To be more clear, this:
contvar1 contvar2 45 3 100 NA 73 5 NA 7 62 NA NA 4 94 6
would become this:
contvar1 contvar1_NA contvar2 contvar2_NA 45 0 3 0 100 0 0 1 73 0 5 0 0 1 7 0 62 0 0 1 0 1 4 0 94 0 6 0
The idea is that, for instance in row 4 above, the coefficient for
contvar1 would be multiplied by
0 and therefore not affect the prediction, but then a separate coefficient for
contvar1_NA would be learned and that would offset the prediction by the appropriate amount.
This intuitively makes sense to me, but I can't find much written about an approach like this (other than this post, which calls it "often useful") so I wanted to see what others thought. It feels to me sort of analogous to what XGBoost does (some discussion here) of learning the appropriate split for a missing value, just like it would for any other value of that variable.
As a sidenote, I'm not 100% tied to using LASSO, but the size of my data makes it not ideal to use something like XGBoost because of both training and prediction time. LASSO is much much faster, especially for predictions (which I have to do a lot of).
Any thoughts are much-appreciated.