I'm interested in studying the impact or importance of each feature on the response variable.
I'm thinking running multiple linear regression with multiple features, and running regression analysis with t-test to test the significance of each regression coefficients. But I've read that in case of multi-collinearity, the result t-test on regression coefficients are unreliable.
Let's say that I initially have 6 features, then I apply PCA with
n_components=6. In the other words, I'm not doing any dimensional reduction. I'm just removing correlations. When the correlation among features are removed with PCA, will the result of t-test on hte new correlation-removed features be reliable?