Questions tagged [boruta]
A feature selection algorithm based on random forest. This tag relates to both the algorithm as well as the R package.
27
questions
1
vote
1
answer
30
views
BorutaPy: All features are classified tentative
I am using boruta_py, Python implementation of the Boruta algorithm, with a random forest estimator.
...
- 263
0
votes
1
answer
106
views
Boruta followed by LightGBM for feature selection
Assume that we have a high-dimensional data with a few samples. We want to select a minimum set of best features from this dataset using LightGBM feature importance. This is because of an external ...
- 1
4
votes
2
answers
117
views
Why is recommended depth of Boruta random forest between 3 and 7?
On the BorutaPy GitHub page, the authors state: "We highly recommend using pruned trees with a depth between 3-7" (https://github.com/scikit-learn-contrib/boruta_py). What justification is ...
- 924
1
vote
1
answer
184
views
How can a feature that when removed, does not affect the model's performance not be declared unimportant?
In the paper on the Boruta algorithm, there is a statement that is unclear to me (highlighted in black).
The all-relevant problem of feature selection is more difficult than usual minimal-optimal one....
- 230
1
vote
0
answers
163
views
Boruta Shadow Feature has the highest Z-Score
I used the Boruta-SHAP algorithm (https://github.com/Ekeany/Boruta-Shap) for feature selection. The algorithm confirmed two features as relevant and rejected the other ones. Still the highest Z_Score ...
- 11
1
vote
0
answers
67
views
Can Boruta be used for inference about which variables matter?
In particular with Boruta variable selection.
If not, what is a better approach?
Edit: In terms of the spectrum from prediction to inference, I am interested in the latter. That is, I am interested in ...
- 2,381
0
votes
0
answers
98
views
Comparing categories of ranked variables
I have three categories (patient factors, surgeon factors, and surgery factors). The goal is to see which category can predict an outcome such as infection.
Each category is made up of 8-12 binary ...
- 1
0
votes
0
answers
130
views
Bivariate analyses vs. Boruta/random forest for removing irrelevant variables prior to penalized regression
I will be using a penalized logistic regression (elastic net) to select variables and their relative importance for predicting an outcome. The goal is to eventually create a risk prediction model, not ...
- 335
4
votes
0
answers
725
views
Boruta Algorithm for Logistic Regression?
Is it okay to use a Boruta algorithm to select features for a logistic regression?
I read several sources, including the source package as well as this site explaining what Boruta does. My ...
- 41
4
votes
1
answer
920
views
Boruta feature selection method
I have a general question about boruta. I know that it is not necessarily going to improve accuracy, as by design it is not meant to do this. However, what about using it before a regularization ...
3
votes
1
answer
646
views
all-relevant feature selection vs minimum optimum feature selection
There are many different ways to selection features in modeling process.
One way is to first select all-relevant features (like Boruta algorithm). And then develop model upon those those selected ...
- 749
1
vote
0
answers
228
views
Problem to get z-score plot with boruta R [closed]
I am working with Boruta R package for selection variable.
I am trying to replicate plots from Maya Gopal and Bhargavi: FEATURE SELECTION FOR YIELD PREDICTION USING BORUTA ALGORITHM (2018).
https://...
2
votes
1
answer
953
views
unstable result using boruta for feature selection
The Boruta algorithm is a wrapper built around the random forest classification algorithm. It tries to capture all the important, interesting features in data with respect to an outcome variable.
I'...
- 749
3
votes
1
answer
4k
views
What does z-score mean in Boruta
The boruta algorithm performs the following steps. (here). Can anyone explain me what exactly the z-score means in this context?
I am referring to point 5 and 6 in the following list. I only know the ...
- 5,093
0
votes
0
answers
138
views
Do I need to specify any particular settings for Boruta feature selection in regression problems?
I trying to solve a linear regression problem and would like to select the most important variables to use. I came across the Boruta package in R.
I am using it as ...
- 289
5
votes
2
answers
2k
views
Can random forest based feature selection method be used for multiple regression in machine learning
I would like to have a good feature selection method for a continuous response variable, over around 100 predictors. I would like to keep my model type as linear multiple regression model, rather than ...
- 2,011
0
votes
0
answers
96
views
Naive Bayes for GPA classification?
So I am doing a project where I have to find a method that uses a lot of both categorical and numerical variables collected from surveys to predict a child's "discretized" GPA values.
For example, ...
- 101
7
votes
2
answers
5k
views
Boruta 'all-relevant' feature selection vs Random Forest 'variables of importance'
Can someone explain the difference between variables of importance from random forest vs all-relevant features from Boruta feature selection?
For example, if one were to build a model (could be any ...
- 282
1
vote
1
answer
718
views
Feature selection step before decision tree?
I want to use rpart (a R package) to build a decision tree model. The data is a high-dimensional expression matrix, with ~50,000 predictors and ~500 samples. The response is a categorical variable.
...
- 206
2
votes
1
answer
1k
views
Boruta feature selection with R
I used Boruta algorithm in R on a data set of ~600 attributes and with a sample of 50K (the original size is 300K).
Using the following parametres:
pValue = 0.05
getImp = getImpRfZ
maxRuns = 11
...
3
votes
1
answer
918
views
Multiple regression or anova or bestglm or forestplot or Boruta
I have data on a continuous health variable and following others: age, gender, height, weight, waist, city and season.
I applied multiple regression and got following output: (age, gender, height, ...
- 9,407
4
votes
1
answer
164
views
Boruta score goes to minus infinity
I'm running the Boruta algorithm with a $179\times 36$ predictor matrix and a numerical response. Most of the variables have a score going to -Inf. Should I ...
- 18.1k
2
votes
1
answer
203
views
Boruta test and naive bayes classification
I am currently using Boruta to test which feature is the most important to be used in my model development. For example, I have 3 features(X,Y,Z).Boruta test give the highest importance is Z. However ...
- 21
9
votes
3
answers
8k
views
Does Boruta feature selection (in R) take into account the correlation between variables?
I am a bit of a novice in R and feature selection, and have tried the Boruta package to select (diminish) my number of variables (n= 40). I thought that this method also took into account the possible ...
- 93
3
votes
1
answer
281
views
k-fold feature selection
I have a data set with 20 K variables. I have tried to select some features via Boruta and FSelector but I could not achieve ...
1
vote
0
answers
507
views
Interpretation of Scree plots and Boruta Outcomes
I have 37 features in my dataset. I used Boruta package in R and according to its analysis, all the features are "important" and should be retained. I examined this result of Boruta and found that if ...
- 257
6
votes
1
answer
840
views
Reference for random forests
I would like to understand how do the Boruta package work. Could you suggest some references for the theoretical aspect of so-called random forests?
Below are two illustrative examples of why am I ...
- 18.1k