Skip to main content

Questions tagged [boruta]

A feature selection algorithm based on random forest. This tag relates to both the algorithm as well as the R package.

Filter by
Sorted by
Tagged with
0 votes
0 answers
39 views

BorutaPy selects different features in different iterations

I came up to something strange. I know Boruta should select everything that is important. I have a dataframe of 200 observations and 2000 features. if I shuffle the order of the features in the ...
Noob Programmer's user avatar
2 votes
2 answers
395 views

BorutaPy: All features are classified tentative

I am using boruta_py, Python implementation of the Boruta algorithm, with a random forest estimator. ...
arilwan's user avatar
  • 273
2 votes
1 answer
550 views

Boruta followed by LightGBM for feature selection

Assume that we have a high-dimensional data with a few samples. We want to select a minimum set of best features from this dataset using LightGBM feature importance. This is because of an external ...
ML Guy's user avatar
  • 21
5 votes
2 answers
320 views

Why is recommended depth of Boruta random forest between 3 and 7?

On the BorutaPy GitHub page, the authors state: "We highly recommend using pruned trees with a depth between 3-7" (https://github.com/scikit-learn-contrib/boruta_py). What justification is ...
David's user avatar
  • 1,276
1 vote
1 answer
395 views

How can a feature that when removed, does not affect the model's performance not be declared unimportant?

In the paper on the Boruta algorithm, there is a statement that is unclear to me (highlighted in black). The all-relevant problem of feature selection is more difficult than usual minimal-optimal one....
Multivac's user avatar
  • 242
1 vote
0 answers
237 views

Boruta Shadow Feature has the highest Z-Score

I used the Boruta-SHAP algorithm (https://github.com/Ekeany/Boruta-Shap) for feature selection. The algorithm confirmed two features as relevant and rejected the other ones. Still the highest Z_Score ...
Bence Repasi's user avatar
1 vote
0 answers
70 views

Can Boruta be used for inference about which variables matter?

In particular with Boruta variable selection. If not, what is a better approach? Edit: In terms of the spectrum from prediction to inference, I am interested in the latter. That is, I am interested in ...
Dave's user avatar
  • 2,631
0 votes
0 answers
118 views

Comparing categories of ranked variables

I have three categories (patient factors, surgeon factors, and surgery factors). The goal is to see which category can predict an outcome such as infection. Each category is made up of 8-12 binary ...
Azswim's user avatar
  • 1
0 votes
0 answers
141 views

Bivariate analyses vs. Boruta/random forest for removing irrelevant variables prior to penalized regression

I will be using a penalized logistic regression (elastic net) to select variables and their relative importance for predicting an outcome. The goal is to eventually create a risk prediction model, not ...
Mark's user avatar
  • 335
4 votes
0 answers
968 views

Boruta Algorithm for Logistic Regression?

Is it okay to use a Boruta algorithm to select features for a logistic regression? I read several sources, including the source package as well as this site explaining what Boruta does. My ...
Eric's user avatar
  • 41
4 votes
1 answer
1k views

Boruta feature selection method

I have a general question about boruta. I know that it is not necessarily going to improve accuracy, as by design it is not meant to do this. However, what about using it before a regularization ...
Aeysha Bhatti's user avatar
3 votes
1 answer
733 views

all-relevant feature selection vs minimum optimum feature selection

There are many different ways to selection features in modeling process. One way is to first select all-relevant features (like Boruta algorithm). And then develop model upon those those selected ...
zesla's user avatar
  • 779
1 vote
0 answers
251 views

Problem to get z-score plot with boruta R [closed]

I am working with Boruta R package for selection variable. I am trying to replicate plots from Maya Gopal and Bhargavi: FEATURE SELECTION FOR YIELD PREDICTION USING BORUTA ALGORITHM (2018). https://...
Rodrigo Javier Badilla Cabrera's user avatar
2 votes
1 answer
1k views

unstable result using boruta for feature selection

The Boruta algorithm is a wrapper built around the random forest classification algorithm. It tries to capture all the important, interesting features in data with respect to an outcome variable. I'...
zesla's user avatar
  • 779
3 votes
1 answer
4k views

What does z-score mean in Boruta

The boruta algorithm performs the following steps. (here). Can anyone explain me what exactly the z-score means in this context? I am referring to point 5 and 6 in the following list. I only know the ...
Ferdi's user avatar
  • 5,179
0 votes
0 answers
150 views

Do I need to specify any particular settings for Boruta feature selection in regression problems?

I trying to solve a linear regression problem and would like to select the most important variables to use. I came across the Boruta package in R. I am using it as ...
user121's user avatar
  • 319
6 votes
2 answers
2k views

Can random forest based feature selection method be used for multiple regression in machine learning

I would like to have a good feature selection method for a continuous response variable, over around 100 predictors. I would like to keep my model type as linear multiple regression model, rather than ...
tiantianchen's user avatar
  • 2,111
0 votes
0 answers
97 views

Naive Bayes for GPA classification?

So I am doing a project where I have to find a method that uses a lot of both categorical and numerical variables collected from surveys to predict a child's "discretized" GPA values. For example, ...
Waht's user avatar
  • 101
8 votes
2 answers
6k views

Boruta 'all-relevant' feature selection vs Random Forest 'variables of importance'

Can someone explain the difference between variables of importance from random forest vs all-relevant features from Boruta feature selection? For example, if one were to build a model (could be any ...
derelict's user avatar
  • 312
1 vote
1 answer
739 views

Feature selection step before decision tree?

I want to use rpart (a R package) to build a decision tree model. The data is a high-dimensional expression matrix, with ~50,000 predictors and ~500 samples. The response is a categorical variable. ...
lovetl2002's user avatar
2 votes
1 answer
2k views

Boruta feature selection with R

I used Boruta algorithm in R on a data set of ~600 attributes and with a sample of 50K (the original size is 300K). Using the following parametres: pValue = 0.05 getImp = getImpRfZ maxRuns = 11 ...
Erik Martinez's user avatar
3 votes
1 answer
959 views

Multiple regression or anova or bestglm or forestplot or Boruta

I have data on a continuous health variable and following others: age, gender, height, weight, waist, city and season. I applied multiple regression and got following output: (age, gender, height, ...
rnso's user avatar
  • 10.1k
4 votes
1 answer
190 views

Boruta score goes to minus infinity

I'm running the Boruta algorithm with a $179\times 36$ predictor matrix and a numerical response. Most of the variables have a score going to -Inf. Should I ...
Stéphane Laurent's user avatar
2 votes
1 answer
220 views

Boruta test and naive bayes classification

I am currently using Boruta to test which feature is the most important to be used in my model development. For example, I have 3 features(X,Y,Z).Boruta test give the highest importance is Z. However ...
JASMIN's user avatar
  • 21
10 votes
3 answers
8k views

Does Boruta feature selection (in R) take into account the correlation between variables?

I am a bit of a novice in R and feature selection, and have tried the Boruta package to select (diminish) my number of variables (n= 40). I thought that this method also took into account the possible ...
Charlotte's user avatar
  • 103
3 votes
1 answer
284 views

k-fold feature selection

I have a data set with 20 K variables. I have tried to select some features via Boruta and FSelector but I could not achieve ...
Ricardo Cristian Ramirez's user avatar
1 vote
0 answers
517 views

Interpretation of Scree plots and Boruta Outcomes

I have 37 features in my dataset. I used Boruta package in R and according to its analysis, all the features are "important" and should be retained. I examined this result of Boruta and found that if ...
Rushdi Shams's user avatar
7 votes
1 answer
869 views

Reference for Boruta and random forests

I would like to understand how do the Boruta package work. Could you suggest some references for the theoretical aspect of so-called random forests? Below are two illustrative examples of why am I ...
Stéphane Laurent's user avatar