1
$\begingroup$

Let's say I have built a boosting tree or neural network and I standardized my features beforehand. When I built my model, I split my data into training, validation, and test sets - each with their own means and standard deviations for normalization. This is based on the recommendation of a book, which says it is important not to mix your data, even when it comes to standardization.

Anyways, let's say I have a final model. Now, I have a new observation and I want to predict the outcome.

What do I do with the new observation? The feature values are obviously not standardized. Since it is one data point, it doesn't have a standard deviation. Are there any issues with prediction if I feed this new single observation into my model? The scaling would be completely off from the ones I fed into the model.

If I need to standardize it, what should my mean and standard deviation be?

$\endgroup$

1 Answer 1

0
$\begingroup$

One usual approach would be to use the partitioned data to locate your most optimal model hypothesis. Once you have found your model, ie choice of algorithm, hyperparameters etc you can train it on the entire dataset.

The train and validation sets are there to simulate an environment of unknown data. So you want to make sure that the process you choose for creating val and test sets mimicks how you will receive new data in the real world.

Another point is that in practice you often have some time dependence in your data and you want to incorporate the latest data in your model. However, this latest data is in this case the data you should use for validation and testing. So in the end you want to include this data in the final model.

Once you have done this you can use your final dataset as a basis for normalisation also for incoming data.

If you have no time dependence this might not be needed and you can use the most robust estimates. Since the train data often is many times larger than the test set, this should be used to provide the estimate.

The key takeaway is to make sure that you treat your val and test sets in a way that realistically reflects how new data will appear in reality. And this will also dictate how you do normalization.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.