2
$\begingroup$

I'm taking a class now and they have contradicted themselves, so I'm looking for some clarification.

Can you standardize your values on the ENTIRE data set or does it have to be done only on train (because of data leakage). What about creating dummy variables, inputing missing values, and hyper parameter tuning.

Originally they said all of these had to be fit on train and then transformed on test. Now they are saying it can be done on the entire data set. Can anyone help me understand which is correct?

$\endgroup$
1
  • 2
    $\begingroup$ Preprocessing is a tiny model. Whatever means and standard deviations are learned from the training set are applied to the test set. Else, data leakage is risked as you've noted. $\endgroup$ – Demetri Pananos Apr 15 at 18:30
4
$\begingroup$

When you transform the data, you're supposed to transform feature values of the training objects, and then use any parameters from that for transforming values of the same features for test objects. (do not transform all the data together). Thus, for mean-zero standardization:

  1. Estimate average and s.d. for feature values in the training set
  2. Standardize the feature values of the training objects.
  3. Apply the averages and s.d. of feature values of training objects to transform feature values of test objects.

Same with normalization in range [0,1]. Dummy(0,1) variables don't need to be transformed. Imputing would follow the same logic -- use the training set to get the imputation information, and apply to the test set.

$\endgroup$
1
  • 2
    $\begingroup$ Agreed. I like to think about it like "production" where there is a continual flow of novel input to the learner. You want the same transform that goes on the training to be applied to operational data, but you can't know what the mean of the infinity of unknown inputs will be. If you knew all the data ever, why use training data? There are slow-moving adaptations that can run on production ingest and improve quality of computer processing over time, but that must be treated gently. $\endgroup$ – EngrStudent Apr 15 at 16:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.