Questions tagged [importance]
The importance of an independent variable or predictor in explaining or predicting the outcome of interest.
393
questions
45
votes
2
answers
64k
views
Measures of variable importance in random forests
I've been playing around with random forests for regression and am having difficulty working out exactly what the two measures of importance mean, and how they should be interpreted.
The ...
- 2,782
36
votes
3
answers
34k
views
Feature importance with dummy variables
I am trying to understand how I can get the feature importance of a categorical variable that has been broken down into dummy variables. I am using scikit-learn which doesn't handle categorical ...
- 1,437
33
votes
4
answers
7k
views
What are variable importance rankings useful for?
I have become somewhat of a nihilist when it comes to variable importance rankings (in the context of multivariate models of all kinds).
Often in the course of my work, I am asked to either assist ...
- 36.1k
27
votes
4
answers
59k
views
Estimating the most important features in a k-means cluster partition
Is there a way to determine which features / variables of the dataset are the most important / dominant within a k-means cluster solution?
- 373
25
votes
4
answers
115k
views
Maximum number of independent variables that can be entered into a multiple regression equation
What is the limit to the number of independent variables one may enter in a multiple regression equation? I have 10 predictors that I would like to examine in terms of their relative contribution to ...
- 409
24
votes
3
answers
43k
views
How to split r-squared between predictor variables in multiple regression?
I have just read a paper in which the authors carried out a multiple regression with two predictors. The overall r-squared value was 0.65. They provided a table which split the r-squared between the ...
- 14.5k
23
votes
2
answers
22k
views
Variable importance from GLMNET
I am looking at using the lasso as a method for selecting features and fitting a predictive model with a binary target. Below is some code I was playing with to try out the method with regularized ...
- 8,780
19
votes
5
answers
7k
views
Understanding which features were most important for logistic regression
I've built a logistic regression classifier that is very accurate on my data. Now I want to understand better why it is working so well. Specifically, I'd like to rank which features are making the ...
- 6,708
19
votes
1
answer
6k
views
In Random Forest, why is a random subset of features chosen at the node level rather than at the tree level? [duplicate]
My Question: Why does random forest consider random subsets of features for splitting at the node level within each tree rather than at the tree level?
Background: This is something of a history ...
- 606
18
votes
3
answers
22k
views
Feature Importance for Linear Regression
Is there a way to find feature importance of linear regression similar to tree algorithms, or even some parameter which is indicative?
I am aware that the coefficients don't necessarily give us the ...
- 363
14
votes
4
answers
3k
views
Comparing importance of different sets of predictors
I was advising a research student with a particular problem, and I was keen to get the input of others on this site.
Context:
The researcher had three types of predictor variables. Each type ...
- 45.5k
14
votes
2
answers
12k
views
Is feature importance in Random Forest useless?
For Random Forests or XGBoost I understand how feature importance is calculated for example using the information gain or decrease in impurity.
In particular in sklearn (and also in other ...
- 1,250
14
votes
1
answer
14k
views
Variable importance in RNN or LSTM
Several method have been devised for accessing or quantifying variable importance (even if only relative to each other) in MLP neural network models:
Connection weights
Garson’s algorithm
Partial ...
- 509
14
votes
2
answers
19k
views
Variable importance randomForest negative values
I am asking myself if it is a good idea to remove those variables with a negative variable importance value ("%IncMSE") in a regression context. And if it gives me a better prediction? What do you ...
- 1,381
14
votes
4
answers
10k
views
Importance of variables in logistic regression
I am probably dealing with a problem that has probably been solved a hundred times before, but I'm not sure where to find the answer.
When using logistic regression, given many features $x_1,...,x_n$ ...
- 348
14
votes
3
answers
6k
views
How can I get feature importance for Gaussian Naive Bayes classifier?
I have a dataset consisting of 4 classes and around 200 features. I have implemented a Gaussian Naive Bayes classifier. I want now calculate the importance of each feature for each pair of classes ...
- 1,814
13
votes
1
answer
5k
views
If feature importance is only computed based on training set, does it mean one should never compute shap values on test set?
If feature importance is only calculated from the training set according to here, does it mean one should never compute shap values on test set? What would it mean if I compute shap values from test ...
- 1,236
13
votes
2
answers
58k
views
How to quantify the Relative Variable Importance in Logistic Regression in terms of p?
Suppose a logistic regression model is used to predict whether an online shopper will purchase a product (outcome: purchase), after he clicked a set of online adverts (predictors: Ad1, Ad2, and Ad3).
...
- 131
12
votes
2
answers
1k
views
Relative variable importance/explained variation from a single model fit
I am seeking a measure of relative variable importance or relative explained variation that will apply to all types of linear and nonlinear regression models and that requires only fitting one model. ...
- 96.5k
10
votes
4
answers
41k
views
How to interpret standardized regression coefficients and p-values in multiple regression?
I've been using R to analyze my data (as shown in example below) and lm.beta from the QuantPsyc package to get the standardized ...
- 103
10
votes
1
answer
5k
views
randomForest and variable importance bug?
I dont get the difference between the rfobject$importance and importance(rfobject) in the MeanDecreaseAccuracy column.
Example:
...
- 1,381
10
votes
1
answer
1k
views
Optimal importance sampling with ratio estimator
We want to approximate the following expectation:
$$\mathbb{E}[h(x)] = \int h(x)\pi(x) dx$$
Where $h(x)$ is an arbitrary function and $\pi(x)$ is a distribution, also for simplicity, let's assume ...
- 1,268
9
votes
1
answer
22k
views
What is variable importance?
Searching this site, I see over 1,000 posts triggered by the search term "variable importance", mostly machine learning related. However, I've never encountered the definition before. Is it a term ...
- 64.2k
9
votes
2
answers
6k
views
Feature Interaction and its applications?
What is a Feature Interaction?
Are Feature Interactions used for Feature Selection or Feature Generation?
- 2,995
9
votes
2
answers
1k
views
Why is the variable importance metric suggested by Breiman specific only to random forests?
In the Random Forest paper they describe a nice way of measuring a variable importance - take your validation data, measure error rate, permute the variable and re-measure error rate.
Question - why ...
- 3,350
9
votes
1
answer
689
views
A statistical test to measure the importance of features?
I'm currently trying to assess importance of the features for my classifier. The situation is the following: first I train my classifier with all of the features I have and tested on a test set . Then ...
- 8,401
8
votes
4
answers
1k
views
Can the χ² test be used without a contingency table?
I thought that the chi squared (χ²) test is to be used when one has an r × c contingency matrix, i.e., when the dependent variables are nonnegative, span the same <...
- 795
8
votes
2
answers
6k
views
Boruta 'all-relevant' feature selection vs Random Forest 'variables of importance'
Can someone explain the difference between variables of importance from random forest vs all-relevant features from Boruta feature selection?
For example, if one were to build a model (could be any ...
- 312
8
votes
1
answer
19k
views
mtry and unbalanced use of predictor variables in Random Forest
I am working on the Random Forest prediction, with the focus on the importance of predictor variables, and have a question regarding understanding of mtry and the actual usage of variables in the ...
- 779
8
votes
1
answer
553
views
Why can a model's SHAP values change on a new dataset?
Background
I'm validating a model and as part of the process I've been calculating SHAP values for different validation datasets.
I've calculated SHAP values for every sample in each dataset taken ...
- 645
8
votes
1
answer
6k
views
Explanatory power of a variable
I have simple linear regression model. What I want to calculate is how "important" each of my input variables are i.e. to make a statement something like this:
"60% of predictive power in this model ...
- 7,281
8
votes
4
answers
2k
views
How to tell which variable is more meaningful when modeling the relationship between several predictors and outcome variable?
I'm facing a problem in which I need to figure out two things:
which predictor, out of several relevant ones, is the most meaningful one in its effect/predictive power over a predicted variable.
the ...
- 207
8
votes
2
answers
2k
views
Hard in calculating predictor‘s Relative Importance for GAM?
Although there is no agreement upon "relative importance for predictors" with (even) linear models (one possible definition: lmg method), I would still want to know whether there are some ...
- 1,039
7
votes
2
answers
3k
views
Do random forest variable importance measures take into account the interactions?
Do random forest measures of variable importance (mean change of accuracy, mean change of Gini index) take the interactions into account? I think I know how we come up with the variable importance ...
- 73
7
votes
2
answers
2k
views
Is it a creditable approach to use Random Forrest Variable importance for causal inference?
I recently ran into a discussion with a college who used Random Forrest Variable importance to discover causal links between some actions of web users and their characteristics. As I come from ...
- 337
7
votes
1
answer
1k
views
how to deal with correlated/colinear features when using Permutation feature importance?
Permutation feature importance (PFI) is a nice way of getting feature importance in black-box models or models where it is difficult to characterise the relationship between the features and the ...
- 1,373
6
votes
3
answers
5k
views
In the R randomForest package for random forest feature selection, how is the dataset split for training and testing?
I'm using the randomForest R package to perform a random forest feature selection. I undestand that, after the execution of the randomForest function, I have to check the importance field, and study ...
- 742
6
votes
2
answers
3k
views
How do I test whether one predictor is significantly better than another? Is Hotelling's T the best option?
Overview:
I want to test if "emotional numbing" is a significantly better predictor of "lower intimate relationship functioning" than "reexperiencing" or "hyperarousal"
It was suggested to me to use ...
- 61
6
votes
2
answers
7k
views
Per cent increase in MSE (%IncMSE) random forests importance measure: why is mean prediction error divided by standard deviation?
Random forests have their variable importance calculated using one of two methods, of which permutation-based importance is considered better. In R's randomForest ...
- 19.8k
6
votes
1
answer
44
views
How to find significant predictors that can differentiate case and control without ML approach?
I have a dataset with more than 70 columns and I have an binary output column.
What I did currently was to explore the dataset by plotting the bar and line graphs for the input variables vs output ...
- 3,322
6
votes
1
answer
273
views
Using ordinal regression to evaluate predictor "importance"?
We've got a construct-likert-scale with an internal (8 items) and an external dimension (6 item) and there is also a 5-point item y assessing the "subjective" perception (How skilled do you think you ...
- 247
6
votes
1
answer
1k
views
Can someone explain the concept of Shapley value masking when working with tabular data and classification problems?
My understanding of a mask in arrays is to have boolean values matching the shape of a query array (n, m) where you would mask the query and perform an operation. For example, this operation to sum ...
- 1,482
6
votes
1
answer
237
views
Once you have used LASSO to generate regression coefficients, is there another step that gives you information about the model?
I've run a LASSO to build a model out of ~60 potential predictors. I'm wondering what the next step is? If there were OLS regression I would find model fit statistics like R2 or AIC. I would also find ...
- 2,651
6
votes
1
answer
94
views
Importance of regressors in time series data
Could anyone recommend bibliography or name some useful methods to analyze which (exogenous) variables are most important in determining the value of a time series?
For context, I have a random time ...
- 728
6
votes
0
answers
2k
views
6
votes
0
answers
7k
views
How to interpret caret's variable importance and feature selection plots? [duplicate]
I am having some problems understanding the variable importance and feature selection graphs from ...
- 295
5
votes
3
answers
8k
views
Normalizing logistic regression coefficients?
With my limited understanding of the logistic regression, I understand that the coefficients in logistic regression are the odds ratios.
Does it make send to normalize them (divide each one over the ...
- 153
5
votes
1
answer
9k
views
Relative importance of predictors in logistic regression
I would like to calculate an estimate (even a very rough one if it is the best I could get) of the relative importance of predictors in a logistic regression, something which can let me tell a common ...
5
votes
1
answer
86
views
Сan a subset of features perform better than the base set
I have a theoretical question..
I have a model, let it be random forest
I take 100 candidate features and train the model
I select all the features that are important from the point of view of the ...
- 269
5
votes
1
answer
2k
views
Using Random Forest variable importance for feature selection
I'm currently trying to convince my colleague that his method of doing feature selection is causing data leakage and I need help doing so.
The method they are using is as follows:
They first run a ...
- 1,548