I am working on this problem where I have 20 odd features (input variables) and two dependent variables. The objective is to find the variance structure of one of the dependent variables. More importantly, I need to find which of the features explain the most amount of variance in the output/dependent variable. I had the following 3 approaches in mind:
- Run a multiple regression on the standardised input variables and see which one has the highest coefficient. Would this tell me that the feature with the highest regression coefficient explains the most amount of variance in the output variable?
- I read some research papers on the decomposition of R-squared, where the statistical software would tell us how the R-squared of the regression is decomposed into various features. I don't know how to implement this on Python or any of its libraries. Could someone help?
- I know that PCA is used for feature selection but I am not completely sure how to get the original features that explain the most amount of variance in the dependent variable using PCA. Does PCA only tell me dimensions along which the data has the highest amount of variance or does it specifically tell me dimensions, and by extension, original features, that explain the variance in the dependent variable? Could anyone guide me on this?
What approach, if any other, should I be following with regards the above objective?