k fold cross validation: nominal predictor level appears in the test data but not the training data I wrote my own cross validation function for model output in R (lm, glm and so on; named MOD_k_fold_r2 in my R package). It works for nominal predictors as well. In a few rare cases, a level of the nominal predictors does not appear in the training data, but does appear in the test data. When this happens, the predict function throws an error because it doesn't know which beta to assign to this new level and therefore it can't predict the Y values for the cases.
Is there any standard way to deal with this? Perhaps assign the beta a 0 coefficient manually? This would require scoring the new cases manually instead of using the predict function I think. Currently, I am instead just disregarding these runs that produce errors and rely on the remaining runs. This is a rare problem for most datasets.
Update
Proposals so far:


*

*Skip runs that use test data that have levels that do not appear in the training data.

*Use stratified sampling to make sure that all training datasets include at least one case with each level.

*If levels are missing, then insert pseudocases with arbitrary datapoints, e.g. mean of all values.

 A: If you are doing one hot encoding without a reference level, then set all encoded features of that test instance to zero. I don't know if that's common practice, but it sounds logical.
If it's an ordered factor, then you will probably transform it to a numeric feature, and the transformation would probably still applies for new levels.
A: You could look into stratified sampling, i.e. constraining your train/test splits so that they have (approximately) the same relative frequencies for your predictor levels. 
However, I think it worth considering whether the current behavior is actually wanted: So random splitting with non-negligible frequency results in sets that do not cover all predictor levels. Can you consider such a set representative for whatever the application is?
I've been working with such small sample sizes and went for stratified splitting. But I insist that thinking hard about the data and the consequences of working with such small samples is at least as necessary as fixing the pure computational error.
A: You definitely want to warn the user when this happens.  But if robustness and ease of use are important, you might consider a preprocessing step that changes the missing nominal values to an arbitrary value that was in the training data.
