Feature engineering before or after scaling?

Question

I am doing feature engineering on a set of features to reduce the size of the dataset. The features can have different scales. E.g, one feature has values that vary between 1000 and 1500, and the other features vary between 0 and 100. One of the tests that I do in feature engineering is to remove one feature that has high correlation with another.

I have tried to scale the data before doing the feature engineering, and also the opposite. In the first use case, I obtain 60 features, and on the latter use case, I obtain 54 features.

Which is the correct way to do it? Should we do feature engineering before or after scaling?

Answer 1

Standardisation can be applied beforehand as well, but typically, it takes place after feature generation. For example,

• your features might have specific meanings, e.g. click through rate (CTR) = clicks / shown ads, and if you scale clicks and ad shown beforehand, you'll use the CTR signal.

• new features can be out of scale, e.g. if you scale $x$ to 0-1 and then take $x^8$ as a new feature, it'll probably in a smaller range than 0-1.

If you choose to apply scaling beforehand, you'd also need to decide whether you're going to apply scaling again after generating new features or not, especially if the new features' ranges are way different than standardised features