Lets say we have a dataset with hundreds of features. Since I'm not really sure whether all these features are good identifiers or not, I think we might need use one or all of the following techniques in the preprocessing step (before feed the data into a classifier):
- Use VIF to test and drop those multi-collinear features
- Use PCA, LDA etc. to create uncorrelated features
- Use function as
SelectFromModelin scikit-learn package to find the most important features
However, when I tried to apply them in a particular problem, I'm a kind of lost,
- In what situation can we make sure that the preprocessing step is enough?
- I think the four step I mentioned above deal with different problems that may happens in the data, so I just wondering if it's appropriate I always concatenate all of them in the preprocessing step?