I have been using several techniques, sometimes in conjunction with one another, in a pre-processing pipeline prior to using the data for supervised machine learning.
I was wondering about the effects of the order that I perform the operations.
Several operations that I have been performing are:
Outlier removal (either removing all instances that contain any feature with a value 3 standard deviations above or below the mean, or windsorizing the feature to the corresponding value)
Downsampling (I either remove instances from the majority class until they equal the minority class or upsample the minority class using SMOTE)
Standardization (Using the z-score for a feature instead of the raw score)
Standardization and outlier removal are done on a within subject basis.
I feel like outlier removal should go before standardization, as the standardization would be effected by any outliers, but outlier removal would still work fine after standardization (since I am computing the z-score to remove outliers anyway)
Downsampling is currently performed last.
Could anyone explain some of the interactions between these, or point me to somewhere that would have some of this information?