# Dealing with missing data in the prediction set only

I have a regression problem where the independent variables are all factors (categorical). I've been looking at the literature on missing data, and so far it all seems concerned with missing training data. I was wondering if there is a standard way of dealing with missing data in the prediction set. That is, you have all the data you need to train, but then you need to be able to make a prediction with only partial data. This must have been a studied problem.

My initial thought is to use an average of the dummy encoded variables, according to how common they are. As a quick example, say we have a three level factor dummy encoded as

level 1: [1 0]
level 2: [0 1]
level 3: [0 0]


Say level $i$ occurs fraction $f_i$ of the time in the training data (so $\sum_i{f_i}=1$), and the regression has the two coefficients $\beta_1$ and $\beta_2$.

Then a missing value in this factor might be estimated as: $$\beta_1*f_1 + \beta_2*f_2 + 0*f_3$$ But given that the "default" level encoding are shared across factors, I'm not sure I'm handling level 3 correctly in this case.

One note is that I can't tell from your description if you are weighting by raw frequencies. If so, you want to divide these by $N$ to get the marginal probabilities instead.
You are right that you are not handling level 3 correctly. The coding scheme that you use in your question set up is known as reference level coding. To use this approach correctly, you need to have an intercept (i.e., $\beta_0$), which estimates the mean of level 3. I suspect you do have such, even though you didn't list it. In this case, you would just add the intercept to your final equation. That is: $$\beta_0\!*\!f_3 + \beta_1\!*\!f_1 + \beta_2\!*\!f_2$$ Note that you are multiplying the intercept (which encodes the reference level) by the marginal probability that the observation is actually the reference level.
• I do have a common intercept $\beta_0$, but I have a number of factors in the model. I was just showing one factor in the example. Is it possible to suppress the intercept with more than one factor? I thought that you could do it for one, but then it broke down with multiple. That would fix my problem, as it would allocate the overall intercept to each factor. – cschmidt Oct 10 '13 at 16:02