# Questions tagged [importance]

The importance of an independent variable or predictor in explaining or predicting the outcome of interest.

160 questions
5 views

### Calculation of Shapley values: usage of empirical mean of omitted (categorical) variables?

Following Molnar (2019, https://christophm.github.io/interpretable-ml-book/shapley.html#fnref39), the Shapley values of a specific input feature $x_j \in \{x_1, ..., x_p\}$ of an ML model $\hat{f}$ is ...
40 views

### Feature selection in xgboost vs GBM in H2O

I am working on a big data set( more than 100 variables) and 30 million observations. I tried to build 100 models with a grid search using both XGBoost and GBM in H2O (Sparkling Water). I realized ...
32 views

### Understanding a Classification Tree applied to the iris dataset

I used the iris dataset for a simple regression tree to see how things work. I am generally interested in two aspects: Why does the tree not use the Sepal data to improve the classification What do ...
22 views

### How to compare multiple weight vectors of different size?

I am facing a statistical problem that I am not sure whether it's solvable. Simply put, I am given multiple weight vectors and here are two examples: $w_1 = [.2, .3, .4, .1]$ for items A, B, C, D, ...
45 views

### How p-value is calculated in random forest when using measure_importance?

I am using random forest for importance analysis of my variables. I use measure_importance() and at the last column, p values are given. I used R. I used randomForest() and RandomForestExplainer() ...
44 views

### error of permutation feature importance [closed]

Feature importance can be calculated by permuting features. But it seems to suffer from instability. To be more accurate about the feature importance, chap 5.5.1 in interpretable machine learning ...
23 views

### ada model- variables overall importance

I have the object ada from a model I didn't train to predict a binary result (I don't have the training set). Ada package was used. And the result are 200 binary trees. I would like to have a ...
32 views

### What is the industry standard way for determining feature “importance”?

My understanding is, there are different tests to run such as ANOVA, Pearson's Correlation, Chi-Square. Choosing these tests is dependent on if the features / responses are categorial / continuous. ...
30 views

### feature importance using forward selection

In the following article the author has correctly mentioned that the "petal" is more important than "sepal" in case of iris data: https://towardsdatascience.com/feature-importance-and-forward-...
37 views

### Distribution of variable importance in r party package [closed]

I have a dataset of 14558 rows and 250 variables. I am trying to solve a classification problem thanks to r party package and the cforest function (which corresponds to a Random Forest). I would like ...
282 views

### Scale dummy variables in logistic regression

Let's say I have a data set that mixes categorical and continuous features and I would like to study the relative importance of each feature in the prediction of a certain class. For that I am using ...
22 views

### How does random forest calculates the importance of the features?

Trying to understand completely how does random forest work and playing with it a bit, I came across the importance() function here on sklearn. This function has ...
38 views

### Importance of regressors in time series data

Could anyone recommend bibliography or name some useful methods to analyze which (exogenous) variables are most important in determining the value of a time series? For context, I have a random time ...
44 views

### Interpreting MeanDecreaseGini graph in R

Using the randomForest package in R, I fitted a customer churn dataset to a random forest model. The first objective was to identify the most important variables in ...
161 views

### remove features that has zero feature importance in random forest

We have 10 features that is pre-selected from domain knowledge. We ran random forest with those features. one of the feature has zero feature importance. My question is: For those features that has ...
101 views

### Variable importance (?) for multivariate time series anomaly detection methods

I'm working on anomaly detection methods for multivariate time series $[\mathbf{x}^{new}_1,\dots,\mathbf{x}^{new}_T]$ where $\mathbf{x}^{new}_{i}$ is $p-$dimensional. I won't go into the details of ...
18 views

### Can we compare odds ratio from one factor to another factor?

For example ${\rm logit}(\pi)=\beta_1 \cdot x_1+ \beta_2 \cdot x_2$, then I find that the odds ratio of $x_2$ is 27, and the odds ratio of $x_1$ is 1.5, can I say that $x_2$ has a more significant ...
40 views

### Workflow design: Random forest regression to see which independent variables are more important

I want to build a very simple random forest for regression (not classification). I have one numeric dependent variable and 11 independent variables (3 of them are numeric and the rest are categorical)....
60 views

### Calculate gini importance or MDI on the OOB data?

"gini importance" or "mean decrease impurity" (MDI) is one of the methods of calculating feature importance in tree models. This downside of this method is that it bias towards variables with more ...
34 views

### How bootstrap sampling cause bias in Random Forest?

I am confused how bootstrap sampling causes bias in the feature selection process of Random Forest. In this article, it says "Obviously, the bootstrap sampling artificially induces an association ...
61 views

### how to deal with correlated/colinear features when using Permutation feature importance?

Permutation feature importance (PFI) is a nice way of getting feature importance in black-box models or models where it is difficult to characterise the relationship between the features and the ...
542 views

### Why does Random Forest variable importance not sum to 100%?

The randomForest package in R has the importance() function to get both node impurity and mean premutation importance for variables. Why, when calculating mean permutation importance, do the results ...
138 views

### Importance of p-value in real world data? Any resources to back up the viewpoint?

P-values denote statistical significance. Real world data is claims data or EMR data or any data that is not clinical trial data basically. Now, there is a world view that p-values do not hold merit ...
274 views

### Random forest variable importance: Mean minimal depth and number of nodes disagree

I'm trying to determine variable importance for a random forest with 8 predictors, and different variable importance measures are telling different stories. The forest was generated in R with ...
73 views

### Retrain random forest with important variables

So I have a classification problem with around 2000 predictors. First I run a random forest model to get important variables. Then I only use those variables (let say the top 30) to run the model ...
610 views

### Feature importances in random forest

I'm using the random forest classifier (RandomForestClassifier) from scikit-learn on a dataset with around 30 features, 3000 ...
489 views

### In Random Forest, why is a random subset of features chosen at the node level rather than at the tree level?

My Question: Why does random forest consider random subsets of features for splitting at the node level within each tree rather than at the tree level? Background: This is something of a history ...
20 views

### which models' variable importance should I trust if their rank are different?

I have a question about vairable importance that is generated from different models, say random forest and logistical regression. For example, if I have two models that are trained on the same ...
215 views

### Alternatives to Random Forest's Feature Importance for choosing best features [closed]

I have already established method using R's Random Forest tools for ranking most important features; for binary classification task. I'm looking for other methods for doing the same task. So that I ...
113 views

### Variable importance in the glmnet

I'm using R for machine learning. The objective is to classify the onset of disease (Two-class). Before conducting a machine learning algorithm, I ran the glmnet (to utilize elastic net) to reduce ...
39 views

### Interpretation of variable importance in Random Forest

I'm currently using Random Forest to train some models and interpret the obtained results. One of the features I want to analyze further, is the variable importance. The thing is I am not familiar on ...
12 views

### Random Forest Importance Grouping - Permuation [R]

I was impressed with While's answer here: Relative importance of a set of predictors in a random forests classification in R I was wondering if anyone has found a package or algorithm to group ...
63 views

### Effect Size/Variable Importance in Pathmodell with metric and binary predictors (using mplus)

i want to calculate the following Path-Model: I got 4 variables refering to a specific place or spot (like a climbin-rock). There are 2 Liking ratings, one cheap/expensive rating and one young/...
147 views

### Does the absolute values of the %IncMSE in Random Forest hint at predictive power of a variable?

I am new to machine learning. I am researching the drivers of plant species richness in woodlands. I have the species richness and a set of predictors, eg, pH, soil organic matter etc for around 100 ...
54 views

### Relative importance of dummy variables

I would like to identify the relative importance of each predictor variable in a regression model. It's simple enough to do if using just numeric independent variables. The "% influence / relative ...
57 views

### How can Random Forest variable importance be smaller for A compared to B when A has higher correlation with the response Y?

Suppose I am fitting a Random Forest model with A-F as my predictors and Y as my response variable. I then calculate variable importance using the permutation method. Why is it possible for variable ...
146 views

### Correct way to determine the (%) contribution of an independent variable to a MLR model

Is the following methodology correct: Fit a multiple linear regression model Obtain the standardized coefficients Sum up the absolute value of all standardized coefficients Divide each individual ...
26 views

### How to compare the grouped random forest importance across different models?

I have a modeling problem where I have a reasonably large number of features (~75) that can be separated into 3 broad classes. These features correspond to field plots where measurements were taken. ...
1k views

### In the R randomForest package for random forest feature selection, how is the dataset split for training and testing?

I'm using the randomForest R package to perform a random forest feature selection. I undestand that, after the execution of the randomForest function, I have to check the importance field, and study ...
36 views

### Comparing calculated associations vs. logistic regression

We are doing an analysis to find some sort of "recommended average training mileage based on race placements" to recommend training mileage to our members to achieve a certain race placement- not to ...
456 views

### How to analyses sensitivity for understanding which variables are the most effect on the predictive model

I have a dataset with 150 observations. The dataset has 9 input parameters and 1 output parameter. I have built a predictive model (Random Forest) using the dataset. And now, I want to know that which ...
363 views

### Is it valid to use random forests for feature selection in a time series problem?

I'm working on a time series problem, with additional predictors. While I'm exploring various ways to approach the problem, one possible way is to turn the time series problem into a supervised ...
173 views

### Calculating variable importance in a multinomial logit model

I am analysing data from a discrete choice experiment from a sample of 1000 responses where a respondent were presented with two cards and had to choose their preferred option. In the survey there ...
160 views

### Interpreting logistic regression output

Suppose I have a logistic regression output as follows. ...
412 views

### Why can variable importance be negative/zero while its correlation with the response variable is high?

I don't have a working example for this, as I'm using a large dataset in R with the ranger package (Random Forest algorithm) I fit a model using the ranger package with predictors $X_1,...,X_k$ and a ...
34 views

### Necessary to train, tune and test if only estimating variable importance?

In medicine the use of regression models may differ slightly from other fields. We usually build regression models from theory and subject matter knowledge. We're typically interested in estimating ...
3k views

### Interpretation of Importance score in Random Forest

Can I interpret the importance scores obtained from Random forest model similar to the Betas from Linear Regression? For example, if I have an equation $$y=\alpha +\beta_1X_1 + \beta_2X_2,$$ a 1 ...
326 views

### How to determine the most important variables when there are differences in variable importance between predictive models

Background: I am running predictive models to see which variables have the most influence over a chosen measurement. In this example I am comparing the models gbm ...