I've got a problem I'm trying to solve at work where we have over 500 features to predict a binary outcomes (buys/ not buy). I'm being asked to throw everything into a PCA and then run a model.
There's a number of reasons I think this is a bad idea, but after reading further, I'm not too clear as to whether I'm correct. First thing is that we're going to run into multicollinearity issues where certain features may be correlated with each other causing a misinterpretation of the results. However, as I read more about PCA, because it creates orthogonal components, this may not really be an issue? I'd love to hear an answer to this.
Second, given a model with 500 features, even if we do run a PCA and end up with 3-4 components, it'll be near impossible to interpret what is actually going on. Is there a solution to this?