Adding one-hot encoded variable to indicate missing values

Question

I have a dataset with both continuous, categorical, and ordinal variables (roughly 1k of them), over 100 Million rows, and a lot of missing values. I am aware of various methods for imputation, but I'm worried these will either be overly biased (mean or mode imputation, etc.) or unfeasible given the size of the data (MI, other modeling imputations).

I am planning on using LASSO (logistic regression with an L1 penalty) to predict a binary response variable.

I have an idea that I would love to hear people's thoughts on: just add a one-hot encoded variable for each variable with any missing values that would indicate a missing value for that variable.

For the categorical variables, I will be one-hot encoding them anyway, so this seems reasonable. For the continuous and ordinal variables, I would replace the missing values with zero, and then put a 1 in corresponding one-hot "missing" indicator variable.

To be more clear, this:

contvar1  contvar2
45        3
100       NA
73        5
NA        7
62        NA
NA        4
94        6

would become this:

contvar1  contvar1_NA  contvar2  contvar2_NA
45        0            3         0
100       0            0         1
73        0            5         0
0         1            7         0
62        0            0         1
0         1            4         0
94        0            6         0

The idea is that, for instance in row 4 above, the coefficient for contvar1 would be multiplied by 0 and therefore not affect the prediction, but then a separate coefficient for contvar1_NA would be learned and that would offset the prediction by the appropriate amount.

This intuitively makes sense to me, but I can't find much written about an approach like this (other than this post, which calls it "often useful") so I wanted to see what others thought. It feels to me sort of analogous to what XGBoost does (some discussion here) of learning the appropriate split for a missing value, just like it would for any other value of that variable.

As a sidenote, I'm not 100% tied to using LASSO, but the size of my data makes it not ideal to use something like XGBoost because of both training and prediction time. LASSO is much much faster, especially for predictions (which I have to do a lot of).

Any thoughts are much-appreciated.

Answer 1

Ok, I have found an answer to this question from talking to someone I know (a novel approach), so I figured I should share in case someone else stumbles onto this.

My methodology makes sense, if the data has been normalized and/or standardized so that the mean is 0. However, if it has not (like in my example data above) then, in addition to adding the extra indicator variable, you need to do mean- or mode-imputation instead of replacing the value with 0. This is the appropriate way to "neutralize" the effect the given coefficient for that observation. That was the fundamental flaw in my thought process.

It still makes sense to add the extra indicator column, because this will (plausibly) account for any effect on the response that is caused by the data missing not-at-random.