The following question is concerning a project I am working on.
Lets say I have a large dataset with 30 features (columns). I would like to build a Binary Classifier. Say that 10 of those features are to do with some financial data and the other 20 features are something else. We are not interested in financial data, even though we know they are strong predictors, and I was told to remove those features.
However this does not seem right to me, removing some features without analysing the collinearity for example seems like a recipe for disaster as I would technically be creating an artificial dataset. If for example a removed feature was the main predictor, and if it affects another feature in a causal way, then the new model can falsely conclude, that feature best predicts the class.
Am I looking at this right?