It seems that because values are missing from a specific range of my target variable, my model performs poorly when predicting samples that are actually in that range. My target variable is
income and my predictors are
years_at_current_company so my model looks something like:
income ~ years_of_school + years_at_current_company
My training dataset has very few samples where
income < $5000. Why such values are not in my training set is not clear to me, but they are not there and are not recoverable.
It turns out my model performs very poorly on new data that contains incomes below 5k. In fact it almost never predicts a value below 5k (which makes sense since I'm using a random forest). I tried a linear model and it performed worse than the random forest.
Is there a way to effectively deal with this problem? Would it help to oversample the few <5k samples that do exist?