# Adding one-hot encoded variable to indicate missing values

I have a dataset with both continuous, categorical, and ordinal variables (roughly 1k of them), over 100 Million rows, and a lot of missing values. I am aware of various methods for imputation, but I'm worried these will either be overly biased (mean or mode imputation, etc.) or unfeasible given the size of the data (MI, other modeling imputations).

I am planning on using LASSO (logistic regression with an L1 penalty) to predict a binary response variable.

I have an idea that I would love to hear people's thoughts on: just add a one-hot encoded variable for each variable with any missing values that would indicate a missing value for that variable.

For the categorical variables, I will be one-hot encoding them anyway, so this seems reasonable. For the continuous and ordinal variables, I would replace the missing values with zero, and then put a 1 in corresponding one-hot "missing" indicator variable.

To be more clear, this:

contvar1  contvar2
45        3
100       NA
73        5
NA        7
62        NA
NA        4
94        6


would become this:

contvar1  contvar1_NA  contvar2  contvar2_NA
45        0            3         0
100       0            0         1
73        0            5         0
0         1            7         0
62        0            0         1
0         1            4         0
94        0            6         0


The idea is that, for instance in row 4 above, the coefficient for contvar1 would be multiplied by 0 and therefore not affect the prediction, but then a separate coefficient for contvar1_NA would be learned and that would offset the prediction by the appropriate amount.

This intuitively makes sense to me, but I can't find much written about an approach like this (other than this post, which calls it "often useful") so I wanted to see what others thought. It feels to me sort of analogous to what XGBoost does (some discussion here) of learning the appropriate split for a missing value, just like it would for any other value of that variable.

As a sidenote, I'm not 100% tied to using LASSO, but the size of my data makes it not ideal to use something like XGBoost because of both training and prediction time. LASSO is much much faster, especially for predictions (which I have to do a lot of).

Any thoughts are much-appreciated.

• Adding an indicator variable for just 1 observation ("one-hot" encoding) is essentially the same as dropping that observation. It leads to fitting just a specific mean for that variable, so its residual will be zero, and it will not influence the fitting of the other parameters. So, just dropping obs with missing will be simpler! and having the same effect. – kjetil b halvorsen Apr 19 '18 at 17:35
• @kjetil sorry, I wasn't clear. I mean to do this for all observations in a given column that are missing. I have updated the example above to make this more clear. – seth127 May 2 '18 at 20:49