2
$\begingroup$

I'm building a classification model to predict some target variable.

I have only one continuous feature (age) that I am interested in scaling. I split my data into train and test sets, I scale this feature separately in both the train and test (using the scale() function in R), and only then do I train the model, using the caret package.

I've been told this is not entirely correct, I should scale this feature in the test set according to the train set, and I'm not sure what this means exactly. How do I scale a feature in the test set according to the train set?

I'm dealing with the test and train as entirely separate datasets so that I won't have information leakage. If this is not the way to do it, how should I apply it in R?

NOTE - I have many continuous variables in the data, however, I want to scale only the age feature.

Here is my code, I'm using gradient boosting as my classification algorithm:

for (i in (1:10)) {
  print(i)
  set.seed(i)
  
  IND = createDataPartition(y = MYData$Target_feature, p=0.8, list = FALSE)
  TRAIN_set = MYData[IND, ]
  TEST_set = MYData[-IND,]
  
  TRAIN_set$age = scale(TRAIN_set$age)
  TEST_set$age = scale(TEST_set$age)
  
  GBMModel <- train(Target_feature~., data = TRAIN_set,
                    method = "gbm",
                    metric="ROC",
                    trControl = ctrlCV,
                    tuneGrid = gbmGRID,
                    verbose = FALSE
                    
  )
  
  
  AUCs_Trn[i] = auc(roc(TRAIN_set$Target_feature,predict(GBMModel,TRAIN_set, type='prob')[,1]))
  AUCs_Tst[i] = auc(roc(TEST_set$Target_feature,predict(GBMModel,TEST_set, type='prob')[,1]))
  
}
$\endgroup$

1 Answer 1

0
$\begingroup$

For example, for standard scaling, you need to use $\mu_{train}, \sigma_{train}$ while applying scaling to the test set. You should not calculate $\mu_{test},\sigma_{test}$.

More generally, the scaling should be done using the parameters estimated from the training set.

In caret, the example in this page shows you how yo use it using the preprocess method for train and test afterwards: https://www.rdocumentation.org/packages/caret/versions/6.0-92/topics/preProcess

$\endgroup$
4
  • $\begingroup$ What if do TRAIN_set$age = scale(TRAIN_set$age) and then TEST_set$age = (TEST_set$age - mean(TRAIN_set$age) ) / sd(TRAIN_set$age) ? doing it manually.. $\endgroup$ Commented Nov 14, 2022 at 21:43
  • $\begingroup$ Correct, it works. $\endgroup$
    – gunes
    Commented Nov 15, 2022 at 5:37
  • $\begingroup$ but if I do that, the values in the train set would be scaled (around zero) and the values in the test set would be much larger since the mean and the sd of the train set are now very small. So for example subtracting the mean of the train set from the test set, would not make a big change at all.. I'm messing something here ?? $\endgroup$ Commented Nov 15, 2022 at 22:55
  • 1
    $\begingroup$ You're not missing. For a moment, I thought you were using unscaled mean and std. You should use unscaled mean/std. If scale function does mean/std scaling, the scaled mean/std would always be 0 and 1 respectlvely anyways. So, save mean/std to some variables before scaling and then use them to scale the test set. $\endgroup$
    – gunes
    Commented Nov 16, 2022 at 6:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.