Why feature scaling only to training set?

Question

I was following the book "Hands-On Machine Learning with Scikit-Learn & TensorFlow" by "Aurelien Geron".

The following remark was made about feature scaling : -

As with all the transformations, it is important to fit the scalers to the training data only, not to the full dataset (including the test set). Only then can you use them to transform the training set and the test set (and new data)

My understanding of the above text is that feature scaling is done only on the training and not on the test set. Is this interpretation correct?

In case yes, what is the rationale behind not using feature scaling for test dataset?

Answer 1

Not quite. You learn the means and standard deviation of the training set, and then:

• Standardize the training set using the training set means and standard deviations.
• Standardize any test set using the training set means and standard deviations.

This is just following the general principle: any thing you learn, must be learned from the model's training data.

Answer 2

To follow up on a comment made to the answer - If you scale the data before train/test split you will get data leakage. Calculating mean/sd of the entire dataset before splitting will result in leakage as the data from each dataset will contain information about the other set of data (through the mean/sd values) and could influence prediction accuracy and overfit.

Answer 3

Data might already be naturally scaled by the experiments and or conventions used (eg predicting percentages). Secondly, you'll have a system with two variables. So, you cannot readily compare to "unscaled" learning with scaled learning.