Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [importance]

The importance of an independent variable or predictor in explaining or predicting the outcome of interest.

0
votes
0answers
15 views

Effect of the presence and absence of predictors to the classification outcome

I have trained a classification RF model and then ranked the predictors based on their contribution (importance) to the model. How can I estimate the effect of each individual predictor in the ...
0
votes
0answers
24 views

How to determine which variables best differentiate a categorial variable

I have a collection of flagship phones, each characterised by several continuous numerical variables: screen size, megapixels, RAM, storage, RRP, etc. I am trying to work out which variables best ...
0
votes
0answers
12 views

Calculating Variable Importance for Feature Selection - PLSR

I have used the plsr() function in R (from the pls package) to predict a Y variable using many X variables (spectral bands) - and am wanting to calculate variable importance (ViP) to begin to reduce ...
0
votes
0answers
20 views

Variable importance when performing zero-inflated Poisson regression in R?

In short, I need to get the importance of the variables after a zero-inflated regression, with all my predictors being dichotomous factors. I tried something like this: ...
2
votes
1answer
106 views

%IncMSE random forests importance measure - why is mean prediction error divided by standard deviation?

Random forests have their variable importance calculated using one of two methods, of which permutation-based importance is considered better. In R's randomForest ...
1
vote
2answers
72 views

How to interpret / metric Lasso regression coefficients

Edited Question, since it was a duplicate I used Matlab to make a lasso model for my data that has 41 predictors and 1 response variable, and perhaps I used more variables that I need too or maybe ...
0
votes
0answers
20 views

How to determine predictor importance at each level of an HLM model

I am working with a Hierarchical Linear Model (HLM). I want to determine which predictors are important at each level of the data. Across different hierarchical levels the fixed effects along with the ...
0
votes
0answers
11 views

How can random forest variable importance of a pooled sample be greater than variable importance of individual samples?

Here's what I have: a data set with 30 different questions measuring customer experience (on a scale from 1-10), and a question measuring "Overall Customer Satisfaction" (Q1). The data set is ...
3
votes
1answer
40 views

Hard in calculating predictor‘s Relative Importance for GAM?

Although there is no agreement upon "relative importance for predictors" with (even) linear models (one possible definition: lmg method), I would still want to know whether there are some acceptable ...
1
vote
1answer
32 views

The importance() in randomForest returns different results, how to interpret this?

the importance has two variables %IncMSE and IncNodePurity, my results for these two are totally different...I'm predicting a player's value, and want to know which attributes are more important for ...
0
votes
0answers
14 views

How to find percentage contribution of each independent variable on outcome variable in R?

I am running a naive bayes classification model in R on 20 independent variables and an outcome (binary) variable. To gain more insight on the results, I would like to find out how much each of the ...
0
votes
0answers
20 views

Interpretation of Feature Importance Statistic

I would like to discuss the managerial implications of the outcomes of a support vector machine model. I assessed the feature importance statistic for the support vector model (package ‘rminer’) and ...
0
votes
0answers
97 views

Variable importance in a GBM

I have build a model with a Gradient Boosting Machine (GBM) and calculated the feature importance. All features are factors. Now, I know which features are most important. However, the features have ...
1
vote
1answer
33 views

Variable Importance for Logistic regression with categorical data?

If I run the logistic regression with X variables containing categorical data. (I do one-hot encoding on categorical data) How do I evaluate the variable importance? Is there any methods or literature ...
0
votes
0answers
132 views

Feature selection in xgboost vs GBM in H2O

I am working on a big data set( more than 100 variables) and 30 million observations. I tried to build 100 models with a grid search using both XGBoost and GBM in H2O (Sparkling Water). I realized ...
0
votes
0answers
80 views

Understanding a Classification Tree applied to the iris dataset

I used the iris dataset for a simple regression tree to see how things work. I am generally interested in two aspects: Why does the tree not use the Sepal data to improve the classification What do ...
1
vote
1answer
24 views

How to compare multiple weight vectors of different size?

I am facing a statistical problem that I am not sure whether it's solvable. Simply put, I am given multiple weight vectors and here are two examples: $w_1 = [.2, .3, .4, .1]$ for items A, B, C, D, ...
2
votes
0answers
80 views

How p-value is calculated in random forest when using measure_importance?

I am using random forest for importance analysis of my variables. I use measure_importance() and at the last column, p values are given. I used R. I used randomForest() and RandomForestExplainer() ...
0
votes
1answer
78 views

error of permutation feature importance [closed]

Feature importance can be calculated by permuting features. But it seems to suffer from instability. To be more accurate about the feature importance, chap 5.5.1 in interpretable machine learning ...
0
votes
0answers
25 views

ada model- variables overall importance

I have the object ada from a model I didn't train to predict a binary result (I don't have the training set). Ada package was used. And the result are 200 binary trees. I would like to have a ...
1
vote
1answer
37 views

What is the industry standard way for determining feature “importance”?

My understanding is, there are different tests to run such as ANOVA, Pearson's Correlation, Chi-Square. Choosing these tests is dependent on if the features / responses are categorial / continuous. ...
1
vote
1answer
45 views

feature importance using forward selection

In the following article the author has correctly mentioned that the "petal" is more important than "sepal" in case of iris data: https://towardsdatascience.com/feature-importance-and-forward-...
1
vote
1answer
60 views

Distribution of variable importance in r party package [closed]

I have a dataset of 14558 rows and 250 variables. I am trying to solve a classification problem thanks to r party package and the cforest function (which corresponds to a Random Forest). I would like ...
4
votes
1answer
342 views

Scale dummy variables in logistic regression

Let's say I have a data set that mixes categorical and continuous features and I would like to study the relative importance of each feature in the prediction of a certain class. For that I am using ...
1
vote
1answer
24 views

How does random forest calculates the importance of the features?

Trying to understand completely how does random forest work and playing with it a bit, I came across the importance() function here on sklearn. This function has ...
3
votes
1answer
39 views

Importance of regressors in time series data

Could anyone recommend bibliography or name some useful methods to analyze which (exogenous) variables are most important in determining the value of a time series? For context, I have a random time ...
0
votes
1answer
103 views

Interpreting MeanDecreaseGini graph in R

Using the randomForest package in R, I fitted a customer churn dataset to a random forest model. The first objective was to identify the most important variables in ...
3
votes
1answer
315 views

remove features that has zero feature importance in random forest

We have 10 features that is pre-selected from domain knowledge. We ran random forest with those features. one of the feature has zero feature importance. My question is: For those features that has ...
1
vote
0answers
162 views

Variable importance (?) for multivariate time series anomaly detection methods

I'm working on anomaly detection methods for multivariate time series $[\mathbf{x}^{new}_1,\dots,\mathbf{x}^{new}_T]$ where $\mathbf{x}^{new}_{i}$ is $p-$dimensional. I won't go into the details of ...
1
vote
0answers
18 views

Can we compare odds ratio from one factor to another factor?

For example ${\rm logit}(\pi)=\beta_1 \cdot x_1+ \beta_2 \cdot x_2$, then I find that the odds ratio of $x_2$ is 27, and the odds ratio of $x_1$ is 1.5, can I say that $x_2$ has a more significant ...
0
votes
1answer
48 views

Workflow design: Random forest regression to see which independent variables are more important

I want to build a very simple random forest for regression (not classification). I have one numeric dependent variable and 11 independent variables (3 of them are numeric and the rest are categorical)....
3
votes
0answers
38 views

How bootstrap sampling cause bias in Random Forest?

I am confused how bootstrap sampling causes bias in the feature selection process of Random Forest. In this article, it says "Obviously, the bootstrap sampling artificially induces an association ...
4
votes
1answer
74 views

how to deal with correlated/colinear features when using Permutation feature importance?

Permutation feature importance (PFI) is a nice way of getting feature importance in black-box models or models where it is difficult to characterise the relationship between the features and the ...
1
vote
1answer
711 views

Why does Random Forest variable importance not sum to 100%?

The randomForest package in R has the importance() function to get both node impurity and mean premutation importance for variables. Why, when calculating mean permutation importance, do the results ...
-3
votes
1answer
179 views

Importance of p-value in real world data? Any resources to back up the viewpoint?

P-values denote statistical significance. Real world data is claims data or EMR data or any data that is not clinical trial data basically. Now, there is a world view that p-values do not hold merit ...
0
votes
0answers
412 views

Random forest variable importance: Mean minimal depth and number of nodes disagree

I'm trying to determine variable importance for a random forest with 8 predictors, and different variable importance measures are telling different stories. The forest was generated in R with ...
1
vote
1answer
86 views

Retrain random forest with important variables

So I have a classification problem with around 2000 predictors. First I run a random forest model to get important variables. Then I only use those variables (let say the top 30) to run the model ...
0
votes
1answer
733 views

Feature importances in random forest

I'm using the random forest classifier (RandomForestClassifier) from scikit-learn on a dataset with around 30 features, 3000 ...
9
votes
1answer
652 views

In Random Forest, why is a random subset of features chosen at the node level rather than at the tree level?

My Question: Why does random forest consider random subsets of features for splitting at the node level within each tree rather than at the tree level? Background: This is something of a history ...
1
vote
2answers
246 views

Alternatives to Random Forest's Feature Importance for choosing best features [closed]

I have already established method using R's Random Forest tools for ranking most important features; for binary classification task. I'm looking for other methods for doing the same task. So that I ...
1
vote
0answers
139 views

Variable importance in the glmnet

I'm using R for machine learning. The objective is to classify the onset of disease (Two-class). Before conducting a machine learning algorithm, I ran the glmnet (to utilize elastic net) to reduce ...
0
votes
0answers
55 views

Interpretation of variable importance in Random Forest

I'm currently using Random Forest to train some models and interpret the obtained results. One of the features I want to analyze further, is the variable importance. The thing is I am not familiar on ...
1
vote
0answers
14 views

Random Forest Importance Grouping - Permuation [R]

I was impressed with While's answer here: Relative importance of a set of predictors in a random forests classification in R I was wondering if anyone has found a package or algorithm to group ...
0
votes
0answers
64 views

Effect Size/Variable Importance in Pathmodell with metric and binary predictors (using mplus)

i want to calculate the following Path-Model: I got 4 variables refering to a specific place or spot (like a climbin-rock). There are 2 Liking ratings, one cheap/expensive rating and one young/...
1
vote
0answers
183 views

Does the absolute values of the %IncMSE in Random Forest hint at predictive power of a variable?

I am new to machine learning. I am researching the drivers of plant species richness in woodlands. I have the species richness and a set of predictors, eg, pH, soil organic matter etc for around 100 ...
1
vote
0answers
63 views

Relative importance of dummy variables

I would like to identify the relative importance of each predictor variable in a regression model. It's simple enough to do if using just numeric independent variables. The "% influence / relative ...
0
votes
1answer
78 views

How can Random Forest variable importance be smaller for A compared to B when A has higher correlation with the response Y?

Suppose I am fitting a Random Forest model with A-F as my predictors and Y as my response variable. I then calculate variable importance using the permutation method. Why is it possible for variable ...
0
votes
1answer
210 views

Correct way to determine the (%) contribution of an independent variable to a MLR model

Is the following methodology correct: Fit a multiple linear regression model Obtain the standardized coefficients Sum up the absolute value of all standardized coefficients Divide each individual ...
0
votes
0answers
27 views

How to compare the grouped random forest importance across different models?

I have a modeling problem where I have a reasonably large number of features (~75) that can be separated into 3 broad classes. These features correspond to field plots where measurements were taken. ...
4
votes
3answers
2k views

In the R randomForest package for random forest feature selection, how is the dataset split for training and testing?

I'm using the randomForest R package to perform a random forest feature selection. I undestand that, after the execution of the randomForest function, I have to check the importance field, and study ...