I have a question regarding imputation I was not able to find an answer to. Any help would be greatly appreciated.

Let's suppose I have a dataset, impute missing values using the median, train a model and test it. The model has good performance. Now I put this model into production.

When I impute missing values for new records that keep getting fed into the model should I use the median of the original dataset or compute the median of the original dataset + the new records?

  • $\begingroup$ Use the median of the original dataset. How else would you apply your model in a sample of size 1? $\endgroup$ – AdamO Jan 10 '18 at 14:47
  • $\begingroup$ I was also leaning towards using the median of the original dataset. Using the median of the original dataset + the one sample would also be a possibility but it does not sound right. Just wanted a second opinion. $\endgroup$ – Rossella Jan 10 '18 at 15:09
  • $\begingroup$ what you will get in that case is a good sequential estimate of the median, and predictions which are difficult to interpret and apply. $\endgroup$ – AdamO Jan 10 '18 at 15:17
  • $\begingroup$ A more general comment: using the median is not good practice for imputation. Better use a Bayesian multiple imputation model, see e.g. stats.stackexchange.com/questions/303722/… $\endgroup$ – tomka Jan 10 '18 at 16:23
  • $\begingroup$ Unfortunately I am strongly limited by what technology I am allowed to use since things need to be productionized really quickly. Python and R are out, I can only use Alteryx at the moment which has none of the advanced imputation methods. $\endgroup$ – Rossella Jan 10 '18 at 16:47

As AdamO suggested, use the median from the original dataset.

The intuition is this: the data you used to develop the model should be large enough and representative to production distribution. Otherwise (production data is dramatically different) the whole model building process will not make any sense. If the model building data is representative and large, why bother to add few new production data?

Let's have a concrete example. Suppose we are building a model to predict housing price and the feature is number of bed rooms. The model building data (we can divide it into training and testing to build the model), has the median of 3 bed room. This numbers should be representative to the housing market overall. (say we had 500K data points /houses to build our model)

Now assume in production time, the first day is a special day and we have luxury 10 houses feed into the model, and these houses has ~8 bed rooms. In second day, that there are 5 houses with missing value.

Intuitively what would we do? To me, it is intuitive / natural to use 3 for imputation, because this number is calculated from a large representative historical data.

  • $\begingroup$ Thanks for the example. That is what I believed as well but just wanted to make sure since it is the first time I have to put a model into production. $\endgroup$ – Rossella Jan 10 '18 at 16:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.