I am working on an assignment in which I am given a set of features and a (continuous) target and I need to find what is the most important factor driving the target.

I thought about several different ways of approaching it:

  1. Looking at Pearson's and/or Spearman's correlation coefficients between each variable and the target.

  2. Normalize the features, fit a linear regression and look at the absolute value of the coefficients.

  3. Fit a Linear Regression and look at the p-values for the coefficients.

  4. Fit a Random Forest and look at the feature importance (Gini score) for each feature.

One extra difficulty is that two of the features are collinear, (but can be combined together in a way that makes perfect sense for the business case).

Are these methods valid? If so, which one do you think it is more valid? Otherwise, is there any standard way to find this relative importance across features?

  • 1
    $\begingroup$ You should describe your target and mention the factors that you are considering. Also this question requires a self-study tag. $\endgroup$ Mar 25, 2017 at 13:16
  • 1
    $\begingroup$ Check out dominance analysis. $\endgroup$ May 23, 2019 at 12:25

1 Answer 1


is there any standard way to find this relative importance across features?

there are all valid methods, features construction and selection always the most time consuming part, try and error, then find the most important features. there's no best method, different by case. But you can try the following paths :

First, try the pearson correlation between features, remove or combine some highly correlated features.

Second, try the random forest to select the most important features, check the results is reasonable. If so, dig these important features, such as visualize the distribution or the correlation with dependent variables, and so on (you may find some outlier or the relationship between variables)

Third, try to use the most important features selected above fit the linear regression, compare the performance(cross validation to get the prediction performance) with random forest.

Fourth, select the best model and most important features to fit that model. You can get the right important feature use the method you mentioned.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.