0
$\begingroup$

I have run into a strange problem. The xgboost model I built was trained using 12 known features to predict 'y'. This was on a clean dataset that had all the features populated. But, now the unknown set on production has only 3 features. The rest of the features are just NULL.

Can I continue to use the same model to predict on production? Or should I re-train the model with only 3 features and then move code back to production?

$\endgroup$
  • 1
    $\begingroup$ Why are the features NULL.in the production data? Nothing seems especially safe without finding that out first. $\endgroup$ – Scortchi - Reinstate Monica May 4 '17 at 8:58
  • $\begingroup$ i figured out why these are null. The fields come from an ERP, and these are optional fields in the ERP. :( So, the dataset I trained upon was on the ones where all these fields were honestly entered by the ERP team (clean data). But, on prod , with all the different ERP systems pushing data, I landed in a soup now. $\endgroup$ – ForeverLearner May 4 '17 at 9:26
  • $\begingroup$ On a ligher note, Human behavior is the most difficult 'Y' value to predict :-D $\endgroup$ – ForeverLearner May 4 '17 at 9:27
  • 1
    $\begingroup$ Is the training data-set in which all fields were entered, going to be representative of the production data-set in which they weren't? If not, even re-training on just those three features (in your original training set) doesn't seem very safe. $\endgroup$ – Scortchi - Reinstate Monica May 4 '17 at 9:36
  • $\begingroup$ The training set is from a source system (ERP) where these fields are mandatory. This system is stable and hence I chose this for training. But, on production there are a lot of source systems, some of which have almost 60% features missing. $\endgroup$ – ForeverLearner May 4 '17 at 9:49
0
$\begingroup$

it's not safe to continue using the 'old' model for predicting a label for such a data. the dependency of models on features are different from a model to another one, but they all depend on features to classify data points. so if you lose some data features, it's not safe to use the 'old' trained model as the classifier.

$\endgroup$
  • $\begingroup$ I will try to retrain the model and come back with the Error-rate. Right now, as you have pointed out, they are very high and no where close to the training numbers. $\endgroup$ – ForeverLearner May 4 '17 at 9:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.