# How do to deal with data missing NOT at random?

It seems that because values are missing from a specific range of my target variable, my model performs poorly when predicting samples that are actually in that range. My target variable is income and my predictors are years_of_school and years_at_current_company so my model looks something like:

income ~ years_of_school + years_at_current_company


My training dataset has very few samples where income < \$5000. Why such values are not in my training set is not clear to me, but they are not there and are not recoverable.

It turns out my model performs very poorly on new data that contains incomes below 5k. In fact it almost never predicts a value below 5k (which makes sense since I'm using a random forest). I tried a linear model and it performed worse than the random forest.

Is there a way to effectively deal with this problem? Would it help to oversample the few <5k samples that do exist?

• This sounds more like truncation than the standard conception of missing data. I added the tag. You might read our wiki (linked). If you have some <5k's from your new data, you could add them to the training set. It's not the kind of statistics I do, but you could try some method of combining multiple models RF & linear for beyond the bounds. – gung - Reinstate Monica Aug 19 '19 at 20:14
• @gung thanks for adding the tag, I'll be sure to check it out – George Aug 20 '19 at 3:46