Categorical Variable in Training Set does not capture all cases

Question

I am training a predictive model on a training data set, which includes zipcode as one of the predictors. Since zipcode is nominal, I treat it as categorical variable and try to dummify it.

The problem is that the training set does not cover all the zipcodes. For instance, training set may only contain (12345,67890,13456) and there are two dummy variables created for the model. In test set, the zipcode field may span much more than those three values.

My initial hunch is that we can code all the uncovered categories as missing. However, this may incur high risk of bias since there could be many uncovered levels.

Replacing all the uncovered zipcodes with the most popular one is also risky because it implicitly enforce incorrect zipcodes.

Right now the less riskier option is to delete the zipcode predictor. Would there be any better solution to this?

Answer 1

Removing ZIP code would be the right thing to do. It does not make sense to get a prediction from a model that has no information about that case. If the data you wish to make a prediction about includes the ZIP code 61801 but your training data did not, the model has no way of knowing the effect this ZIP code had on the response.