Imagine you have two different datasets each having a feature representing people's ages. One dataset is gathered from teenagers the other is from elderly people.
# Feature 1: ages_1 = [15, 15, 16, 17, 17, 18, 19, 19] # Feature 2: ages_2 = [75, 75, 76, 77, 77, 78, 79, 79]
If we scale these two features, we'll get exactly the same vector, meaning that we'll lose the information that they represent two completely different age groups. Furthermore, unless we store the parameters of the transformation (min/max or mean/std), this information will be irretrievable.
Another consequence of feature scaling is that features lose their interpretability. For example an age of 0.87 (after scaling) would just mean that it is belongs to one of the oldest people in the dataset (could be 18 in the first case or 78 in the second - there is no way we can tell).
Given that we have a lot to lose by scaling the features, why is feature scaling so popular in Machine Learning?