I am working on an assignment in which I am given a set of features and a (continuous) target and I need to find what is the most important factor driving the target.
I thought about several different ways of approaching it:
Looking at Pearson's and/or Spearman's correlation coefficients between each variable and the target.
Normalize the features, fit a linear regression and look at the absolute value of the coefficients.
Fit a Linear Regression and look at the p-values for the coefficients.
- Fit a Random Forest and look at the feature importance (Gini score) for each feature.
One extra difficulty is that two of the features are collinear, (but can be combined together in a way that makes perfect sense for the business case).
Are these methods valid? If so, which one do you think it is more valid? Otherwise, is there any standard way to find this relative importance across features?