Lets say we have a dataset with hundreds of features. Since I'm not really sure whether all these features are good identifiers or not, I think we might need use one or all of the following techniques in the preprocessing step (before feed the data into a classifier):

• Use VIF to test and drop those multi-collinear features
• Use PCA, LDA etc. to create uncorrelated features
• Use function as SelectFromModel in scikit-learn package to find the most important features
• Standardise

However, when I tried to apply them in a particular problem, I'm a kind of lost,

1. In what situation can we make sure that the preprocessing step is enough?
2. I think the four step I mentioned above deal with different problems that may happens in the data, so I just wondering if it's appropriate I always concatenate all of them in the preprocessing step?

Standardizing is a technical trick. It speeds up convergence. It doesn't change anything fundamentally in a reasonable model. Think of using 100mm, 10cm or 1m as a length of the box. Would a model depend on the unit of measure? A sensible model will not.

PCA is used to get orthogonal factors, but not in the context of machine learning, usually. In ML it's most common application is dimensionality reduction. It's a reasonable approach to highly correlated data, but in many cases autoencoder could be preferable. So, it's not a must do step.

I never used VIF even in regression. Multi collinearity is not a big issue in machine learning. In severe cases where you have loads of almost identical variables, it can help with performance to remove collinear variables.

If you're learning ML I wouldn't bother too much about feature engineering in the beginning. Just feed all variables into the model and see what happens.