2
$\begingroup$

In algorithms that perform better with data normalization or deep learning problems such as classification, how normalizing data does not bias our algorithm? I mean, in training or even testing, we can normalize data and columns with very big values compared to others, will become like them. But, in prediction for one query, how does algorithm can predicts correctly without any bias, with such a big value compared to others? Does it even make sense or I am missing sth here?

$\endgroup$

1 Answer 1

1
$\begingroup$

You apply that same normalization transformation to the new data. For instance, if you have a feature that you normalize by subtracting $40$ and dividing by $3$, then to the each new observation of that feature, subtract $40$ and divide by $3$.

EDIT

To respond to a comment:

Ok thanks. in this case, have 3 questions. First: should I use same normalizer or standard scaler on both training and test data? should I do it on training and then use it for test? Or should I use for each of them separately? thirdly, how should I know how I normalized it so that I could use the same on new query for prediction?

Consider the normalizer or scaler to be part of training the model. Whatever you learn about normalizing or scaling the data from the training data is what you do to the rest of the data. That is, if you want to transform to have zero mean, if the training data have a mean of $11$ and the test data have a mean of $12$, you subtract $11$, not $12$. To learn the normalizing or scaling values from the test data represents a data leak.

How you get the values will depend on the exact software functions you call. However, machine learning packages (such as sklearn) have functionality to learn preprocessing steps (such as normalization) from the training data and then apply those results to new data, so you never need to see the exact values being used.

$\endgroup$
2
  • $\begingroup$ Ok thanks. in this case, have 3 questions. First: should I use same normalizer or standard scaler on both training and test data? should I do it on training and then use it for test? Or should I use for each of them separately? thirdly, how should I know how I normalized it so that I could use the same on new query for prediction? $\endgroup$
    – AliM
    Commented Jun 21, 2023 at 17:42
  • $\begingroup$ @AliM I have edited my answer to respond. $\endgroup$
    – Dave
    Commented Jun 21, 2023 at 18:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.