I am training a predictive model on a training data set, which includes zipcode as one of the predictors. Since zipcode is nominal, I treat it as categorical variable and try to dummify it.
The problem is that the training set does not cover all the zipcodes. For instance, training set may only contain (12345,67890,13456) and there are two dummy variables created for the model. In test set, the zipcode field may span much more than those three values.
My initial hunch is that we can code all the uncovered categories as missing. However, this may incur high risk of bias since there could be many uncovered levels.
Replacing all the uncovered zipcodes with the most popular one is also risky because it implicitly enforce incorrect zipcodes.
Right now the less riskier option is to delete the zipcode predictor. Would there be any better solution to this?