Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [importance]

The importance of an independent variable or predictor in explaining or predicting the outcome of interest.

1
vote
1answer
24 views

What is the industry standard way for determining feature “importance”?

My understanding is, there are different tests to run such as ANOVA, Pearson's Correlation, Chi-Square. Choosing these tests is dependent on if the features / responses are categorial / continuous. ...
1
vote
1answer
27 views

feature importance using forward selection

In the following article the author has correctly mentioned that the "petal" is more important than "sepal" in case of iris data: https://towardsdatascience.com/feature-importance-and-forward-...
1
vote
1answer
22 views

Distribution of variable importance in r party package [closed]

I have a dataset of 14558 rows and 250 variables. I am trying to solve a classification problem thanks to r party package and the cforest function (which corresponds to a Random Forest). I would like ...
4
votes
1answer
240 views

Scale dummy variables in logistic regression

Let's say I have a data set that mixes categorical and continuous features and I would like to study the relative importance of each feature in the prediction of a certain class. For that I am using ...
0
votes
1answer
19 views

How does random forest calculates the importance of the features?

Trying to understand completely how does random forest work and playing with it a bit, I came across the importance() function here on sklearn. This function has ...
3
votes
1answer
37 views

Importance of regressors in time series data

Could anyone recommend bibliography or name some useful methods to analyze which (exogenous) variables are most important in determining the value of a time series? For context, I have a random time ...
0
votes
1answer
26 views

Interpreting MeanDecreaseGini graph in R

Using the randomForest package in R, I fitted a customer churn dataset to a random forest model. The first objective was to identify the most important variables in ...
3
votes
1answer
63 views

remove features that has zero feature importance in random forest

We have 10 features that is pre-selected from domain knowledge. We ran random forest with those features. one of the feature has zero feature importance. My question is: For those features that has ...
0
votes
0answers
22 views

Mclust and Random Forest

I have used Mclust for clustering my population (n=39) using a much larger set of features which has been obtained by merging two sets of highly correlated ...
1
vote
1answer
48 views

Variable importance (?) for multivariate time series anomaly detection methods

I'm working on anomaly detection methods for multivariate time series $[\mathbf{x}^{new}_1,\dots,\mathbf{x}^{new}_T]$ where $\mathbf{x}^{new}_{i}$ is $p-$dimensional. I won't go into the details of ...
0
votes
0answers
27 views

Hierarchical multiple regression and importance of individual predictors across multiple analyses

I'm currently reading an empirical study using regression and want to determine whether to consider it a trustworthy, well-analysed piece of literature. I'm more familiar with ANOVA models, so I was ...
1
vote
0answers
18 views

Can we compare odds ratio from one factor to another factor?

For example ${\rm logit}(\pi)=\beta_1 \cdot x_1+ \beta_2 \cdot x_2$, then I find that the odds ratio of $x_2$ is 27, and the odds ratio of $x_1$ is 1.5, can I say that $x_2$ has a more significant ...
0
votes
1answer
33 views

Workflow design: Random forest regression to see which independent variables are more important

I want to build a very simple random forest for regression (not classification). I have one numeric dependent variable and 11 independent variables (3 of them are numeric and the rest are categorical)....
0
votes
0answers
26 views

Calculate gini importance or MDI on the OOB data?

"gini importance" or "mean decrease impurity" (MDI) is one of the methods of calculating feature importance in tree models. This downside of this method is that it bias towards variables with more ...
3
votes
0answers
29 views

How bootstrap sampling cause bias in Random Forest?

I am confused how bootstrap sampling causes bias in the feature selection process of Random Forest. In this article, it says "Obviously, the bootstrap sampling artificially induces an association ...
4
votes
1answer
49 views

how to deal with correlated/colinear features when using Permutation feature importance?

Permutation feature importance (PFI) is a nice way of getting feature importance in black-box models or models where it is difficult to characterise the relationship between the features and the ...
1
vote
1answer
335 views

Why does Random Forest variable importance not sum to 100%?

The randomForest package in R has the importance() function to get both node impurity and mean premutation importance for variables. Why, when calculating mean permutation importance, do the results ...
-3
votes
1answer
125 views

Importance of p-value in real world data? Any resources to back up the viewpoint?

P-values denote statistical significance. Real world data is claims data or EMR data or any data that is not clinical trial data basically. Now, there is a world view that p-values do not hold merit ...
0
votes
0answers
191 views

Random forest variable importance: Mean minimal depth and number of nodes disagree

I'm trying to determine variable importance for a random forest with 8 predictors, and different variable importance measures are telling different stories. The forest was generated in R with ...
1
vote
1answer
58 views

Retrain random forest with important variables

So I have a classification problem with around 2000 predictors. First I run a random forest model to get important variables. Then I only use those variables (let say the top 30) to run the model ...
0
votes
1answer
465 views

Feature importances in random forest

I'm using the random forest classifier (RandomForestClassifier) from scikit-learn on a dataset with around 30 features, 3000 ...
7
votes
0answers
320 views

In Random Forest, why is a random subset of features chosen at the node level rather than at the tree level?

My Question: Why does random forest consider random subsets of features for splitting at the node level within each tree rather than at the tree level? Background: This is something of a history ...
0
votes
0answers
19 views

which models' variable importance should I trust if their rank are different?

I have a question about vairable importance that is generated from different models, say random forest and logistical regression. For example, if I have two models that are trained on the same ...
1
vote
2answers
167 views

Alternatives to Random Forest's Feature Importance for choosing best features [closed]

I have already established method using R's Random Forest tools for ranking most important features; for binary classification task. I'm looking for other methods for doing the same task. So that I ...
1
vote
0answers
83 views

Variable importance in the glmnet

I'm using R for machine learning. The objective is to classify the onset of disease (Two-class). Before conducting a machine learning algorithm, I ran the glmnet (to utilize elastic net) to reduce ...
0
votes
0answers
33 views

Interpretation of variable importance in Random Forest

I'm currently using Random Forest to train some models and interpret the obtained results. One of the features I want to analyze further, is the variable importance. The thing is I am not familiar on ...
1
vote
0answers
11 views

Random Forest Importance Grouping - Permuation [R]

I was impressed with While's answer here: Relative importance of a set of predictors in a random forests classification in R I was wondering if anyone has found a package or algorithm to group ...
0
votes
0answers
59 views

Effect Size/Variable Importance in Pathmodell with metric and binary predictors (using mplus)

i want to calculate the following Path-Model: I got 4 variables refering to a specific place or spot (like a climbin-rock). There are 2 Liking ratings, one cheap/expensive rating and one young/...
0
votes
0answers
117 views

Does the absolute values of the %IncMSE in Random Forest hint at predictive power of a variable?

I am new to machine learning. I am researching the drivers of plant species richness in woodlands. I have the species richness and a set of predictors, eg, pH, soil organic matter etc for around 100 ...
1
vote
0answers
44 views

Relative importance of dummy variables

I would like to identify the relative importance of each predictor variable in a regression model. It's simple enough to do if using just numeric independent variables. The "% influence / relative ...
0
votes
1answer
42 views

How can Random Forest variable importance be smaller for A compared to B when A has higher correlation with the response Y?

Suppose I am fitting a Random Forest model with A-F as my predictors and Y as my response variable. I then calculate variable importance using the permutation method. Why is it possible for variable ...
0
votes
1answer
102 views

Correct way to determine the (%) contribution of an independent variable to a MLR model

Is the following methodology correct: Fit a multiple linear regression model Obtain the standardized coefficients Sum up the absolute value of all standardized coefficients Divide each individual ...
0
votes
0answers
24 views

How to compare the grouped random forest importance across different models?

I have a modeling problem where I have a reasonably large number of features (~75) that can be separated into 3 broad classes. These features correspond to field plots where measurements were taken. ...
3
votes
3answers
1k views

In the R randomForest package for random forest feature selection, how is the dataset split for training and testing?

I'm using the randomForest R package to perform a random forest feature selection. I undestand that, after the execution of the randomForest function, I have to check the importance field, and study ...
1
vote
0answers
36 views

Comparing calculated associations vs. logistic regression

We are doing an analysis to find some sort of "recommended average training mileage based on race placements" to recommend training mileage to our members to achieve a certain race placement- not to ...
1
vote
2answers
309 views

How to analyses sensitivity for understanding which variables are the most effect on the predictive model

I have a dataset with 150 observations. The dataset has 9 input parameters and 1 output parameter. I have built a predictive model (Random Forest) using the dataset. And now, I want to know that which ...
2
votes
0answers
334 views

Is it valid to use random forests for feature selection in a time series problem?

I'm working on a time series problem, with additional predictors. While I'm exploring various ways to approach the problem, one possible way is to turn the time series problem into a supervised ...
1
vote
1answer
142 views

Calculating variable importance in a multinomial logit model

I am analysing data from a discrete choice experiment from a sample of 1000 responses where a respondent were presented with two cards and had to choose their preferred option. In the survey there ...
0
votes
2answers
153 views

Interpreting logistic regression output

Suppose I have a logistic regression output as follows. ...
0
votes
1answer
351 views

Why can variable importance be negative/zero while its correlation with the response variable is high?

I don't have a working example for this, as I'm using a large dataset in R with the ranger package (Random Forest algorithm) I fit a model using the ranger package with predictors $X_1,...,X_k$ and a ...
1
vote
1answer
29 views

Necessary to train, tune and test if only estimating variable importance?

In medicine the use of regression models may differ slightly from other fields. We usually build regression models from theory and subject matter knowledge. We're typically interested in estimating ...
0
votes
0answers
193 views

Weka Attribute Selection Merit and Rank doesn't match

I'm doing some attribute selection with Weka, particularly using SubsetEval and Greedy Stepwise Search Method. After the routine, I see that the output is quite strange with one attribute is ranked ...
1
vote
1answer
2k views

Interpretation of Importance score in Random Forest

Can I interpret the importance scores obtained from Random forest model similar to the Betas from Linear Regression? For example, if I have an equation $$ y=\alpha +\beta_1X_1 + \beta_2X_2,$$ a 1 ...
5
votes
0answers
270 views

How to determine the most important variables when there are differences in variable importance between predictive models

Background: I am running predictive models to see which variables have the most influence over a chosen measurement. In this example I am comparing the models gbm ...
0
votes
0answers
89 views

How to measure the effect of each covariate in GLM

I have a GLM model with two variables (Z1 and Z2). I have also used some basis functions to estimate these variables. In other words, Z1 is substituted by 10 basis functions and Z2 with 12 basis ...
1
vote
2answers
220 views

Is it a creditable approach to use Random Forrest Variable importance for causal inference?

I recently ran into a discussion with a college who used Random Forrest Variable importance to discover causal links between some actions of web users and their characteristics. As I come from ...
1
vote
0answers
133 views

Identify important interactions terms in SVM with polynomial kernel

I have an SVM model with polynomial kernel. I have output of feature weights, but they are all the individual features. Is there a way to know which interactions between individual features are most ...
0
votes
2answers
40 views

Learning weights for variables based on cluster assignments

I need an approach to assign weights to variables based on the cluster. I am not trying to learn the clusters, as I already have them, but I would just like to now understand how "important" each ...
1
vote
1answer
46 views

Which rating variable is affecting the main rating?

I have 3 types of rating, the restaurant rating (main rating), food rating, and service rating. Each of these ratings go from 0 to 2. I would like to know if it's possible to tell if a restaurant is ...
0
votes
0answers
73 views

Strange thing using XGBoost in Classification

I'm working on the Loan Prediction III contest on Analytics Vidhya. The dataset is composed this way: ...