I'm interested in studying the impact or importance of each feature on the response variable.

I'm thinking running multiple linear regression with multiple features, and running regression analysis with t-test to test the significance of each regression coefficients. But I've read that in case of multi-collinearity, the result t-test on regression coefficients are unreliable.

Let's say that I initially have 6 features, then I apply PCA with n_components=6. In the other words, I'm not doing any dimensional reduction. I'm just removing correlations. When the correlation among features are removed with PCA, will the result of t-test on hte new correlation-removed features be reliable?

  • $\begingroup$ yes it will. but those variables won't be the same as before. how much correlated are your variables? people tend to over-worry about collinearity $\endgroup$ – carlo Nov 7 '19 at 20:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.