# Cross Validation with Preprocessing (Normalization, Discretization, Feature Selection)

I am now trying to evaluate my model with cross validation. My dataset contains some numeric and nominal attributes.

Here, I carry out the following data preprocessing tasks:

A. Normalization: Min-Max Normalization (to [0,1])

B. Discretization: Supervised Discretization (Fayyad-Irani), creating bins with some supervised technique (using class label info)

C. Attribute Selection: Correlation based Feature Selection Method

Actually, I first tried to do preprocessing for the entire set, obtaining preprocessed & reduced dataset, then evaluated my model with the dataset through 10-fold cross validation.

However, I found that the way is commonly (miss)used but not good, optimal because of cheating for the test set.

Hence, I am trying to do preprocessing within cross validation. (Yes, preprocessing for only training set for each fold!)

Here, I have a question. As I know, I think using Normalization Filter obtained from training set should be used for the test set. However, if the range of numeric values in the test set is not covered by that in the training set... Then, the normalization result does not create [0,1] range for the test set. For example, if training set has the range from 30 to 50, but test set has the range from 10 to 100, then the normalization filter obtained from the training set looks not appropriate for the test set.

In this situation, how should I do?

(Plus) Is it acceptable way to do only Normalization and Discretization for the entire set, and do feature selection within the cross validation job?

• Beware that the Normalization and Discretization should be constructed so that any thinkable (as in new data) observation still is properly normalizable / discretizable. For example if you want to force your variable into the [0, 1] range, 0 should equal to the population minimum and 1 to the population maximum, not the sample min/max. Doing this you will avoid the problem you mention in the question. This will also result in a fixed transformation (independent of your observations) and can thus be done ourside of the CV-Step. – AlexR Aug 10 '16 at 7:37
• If you do not have to perform min/max scaling, standardizing your features (i.e. removing the mean and dividing by the standard deviation) within each fold would also solve this issue. – cbrnr Jul 13 '17 at 10:04

So the scaling procedure will still be able to work in this scenario. The sklearn min-max scaler transforms data using two variables derived from the data, namely a scaling factor, equal to 1/range(trainingData) and a shift parameter, equal to -min(trainingData)*scalingFactor. It then scales by multiplying by the scaling factor, and then adding the shift parameter. When it comes to the test data, it again simply multiples by the scaling factor and adds the shift parameter. If the range of the test set is larger than that of the training set, this will naturally produce values less than 0 and larger than 1. Ideally, as @AlexR mentioned we would have the population min and mean to perform our scaling with. However, almost always that information will not be available, and using he sample min and max is often our best estimate.