Scaling, normalization and grid search

Question

I have a doubt about scaling and normalization of training data with grid search, where the score of each node is computed with k-fold cross validation. In particular, should training and scaling be independently computed for each k execution on each node, or only one time before the whole grid search?

In other words, should data be scaled and normalized separately for each candidate?

Thanks!

Answer 1

You should always scale based on the train data only, otherwise - you are leaking "knowledge" to the test set.

Therefore, in K-fold CV you should do k normalizations.