Dealing with missing data in the prediction set only

Question

I have a regression problem where the independent variables are all factors (categorical). I've been looking at the literature on missing data, and so far it all seems concerned with missing training data. I was wondering if there is a standard way of dealing with missing data in the prediction set. That is, you have all the data you need to train, but then you need to be able to make a prediction with only partial data. This must have been a studied problem.

My initial thought is to use an average of the dummy encoded variables, according to how common they are. As a quick example, say we have a three level factor dummy encoded as

level 1: [1 0]
level 2: [0 1]
level 3: [0 0]

Say level $i$ occurs fraction $f_i$ of the time in the training data (so $\sum_i{f_i}=1$), and the regression has the two coefficients $\beta_1$ and $\beta_2$.

Then a missing value in this factor might be estimated as: $$ \beta_1*f_1 + \beta_2*f_2 + 0*f_3 $$ But given that the "default" level encoding are shared across factors, I'm not sure I'm handling level 3 correctly in this case.

Answer 1

(I'll let someone else address the estimation of the missing data. You may want to directly model the probability that the observation is each level of the unknown factor using knowledge of other covariate values, and possibly outside information, e.g., priors etc. There are strategies such as propensity scores that you might be able to use for this type of thing. However, at first glance your approach looks reasonable to me.)

One note is that I can't tell from your description if you are weighting by raw frequencies. If so, you want to divide these by $N$ to get the marginal probabilities instead.

You are right that you are not handling level 3 correctly. The coding scheme that you use in your question set up is known as reference level coding. To use this approach correctly, you need to have an intercept (i.e., $\beta_0$), which estimates the mean of level 3. I suspect you do have such, even though you didn't list it. In this case, you would just add the intercept to your final equation. That is: $$ \beta_0\!*\!f_3 + \beta_1\!*\!f_1 + \beta_2\!*\!f_2 $$ Note that you are multiplying the intercept (which encodes the reference level) by the marginal probability that the observation is actually the reference level.