3
votes
1answer
47 views

Finding interactions using randomForest

I am trying to use randomForest in R to find interaction terms to add to a model. My plan was to fit trees with maxnodes=4 (two ...
0
votes
0answers
52 views

Random Forests and Feature selection [duplicate]

First, I split my training set into 10 parts. 9 parts of it, I use as LS and the other one as TS. I now want to do feature selection, so I do feature selection on 9 parts. I use Random Forest to do ...
0
votes
1answer
57 views

Random forest importance differs between rf$importance and importance()

My model is working ok (the AUC is 0.7) but the importances from a randomForest run for my binary classification problem differ depending on how I retrieve them. Is ...
0
votes
2answers
416 views

Random Forest: IncNodePurity and Feature Selection for Binary Logistic Regression

After creating a Random Forest object using randomForest with around 500 candidate variables, I used importance(object) to ...
2
votes
1answer
140 views

Combining Exploratory Factor Analysis with Random Forest for Binary Logistic Regression Feature Selection

For those of you familiar with Exploratory Factor Analysis (EFA) and Random Forest (RF), I have recently had an idea of combining these two methods to reduce the number of potential predictor ...
5
votes
2answers
1k views

Feature selection with Random Forests

I have a dataset with mostly financial variables (120 features, 4k examples) which are mostly highly correlated and very noisy (technical indicators, for example) so I would like to select about max ...
3
votes
1answer
203 views

Using Mutual Information for Binary Logistic Regression Variable Selection

In addition to proc varclus, randomForest, and assessing multicollinearity among potential predictor variables, I am seeking ...
2
votes
0answers
57 views

Out-of-bag estimate biased by correlated features

I have a data set with a small number of samples (322) and a large number of features (318.976). My data consists of images, and I want to train a binary classifier. Since I have such a small amount ...
0
votes
1answer
198 views

What is a good Gini decrease cutoff for feature inclusion based upon random forests?

I am using random forests to try and determine variable importance as part of feature selection for a model I'm working on, and while I can get ranked variable importance by mean decrease in Gini from ...
3
votes
1answer
323 views

Can we use random forest for classification in combination with distance matrix between classes?

With a colleague, we are working on a dataset containing ~5000 continuous variables for 120 individuals belonging to 8 classes. We want to estimate the relative importance of each variable to explain ...
0
votes
1answer
119 views

Relative importance weight with cforest

I am new in using RF. I want to use it to compute the relative importance of the features. I found the weight is very small ("party" package, cforest). Is there anyway to get these weights in a range ...
3
votes
1answer
182 views

Variable importance randomForest negative values

I am asking myself if it is a good idea to remove those variables with a negative variable importance value ("%IncMSE") in a regression context. And if it gives me a better prediction? What do you ...
-1
votes
1answer
150 views

Random forest like procedure for regression or other statistical models

I'm wondering if there exist methods similar to one used in random forest algorithm - I mean taking simultaneously bootstrap sample and random subset of features, then building statistisal model. Have ...
1
vote
0answers
390 views

How to select the best variables by RandomForest in R?

I have a table of mRNA levels of my target gene and it's transcription factors in many different condition. What I want to do is to select the most important ...
1
vote
0answers
207 views

Feature importance

The extremely randomized trees classifier (scikitlearn) provides a (multivariate) feature importance measurement Ensemble methods/feature importance evaluation. For each feature, the classifier ...
1
vote
1answer
155 views

highly correlated features and high ranking

I am classifying different texts and I wondering about some features that are highly correlated. I have 49 features. Some features are absolute counters (integers) but most features are relative ...
4
votes
2answers
1k views

Number of trees for Random Forest optimization using recursive feature elimination

How many trees would you suggest to pick to perform recursive feature elimination (RFE) in order to optimize Random Forest classifier (for binary classification problem). My dataset is very ...
2
votes
0answers
83 views

Random forest like techniques (bagging, random feature subset) for SGD methods

Are there any well-known results/tools/literature on using bagging and random feature subset selection for regression or SGD-based methods?
7
votes
4answers
1k views

Low classification accuracy, what to do next?

So, I'm a newbie in ML field and I try to do some classification. My goal is to predict the outcome of a sport event. I've gathered some historical data and now try to train a classifier. I got around ...
5
votes
1answer
681 views

Feature selection and parameter tuning with caret for random forest

I have data with a few thousand features and I want to do recursive feature selection (RFE) to remove uninformative ones. I do this with caret and RFE. However, I started thinking, if I want to get ...
2
votes
1answer
270 views

Are randomForest variable importance values comparable across same variables on different dates?

Are randomForest variable importance comparable across same variables on different dates? I have a data array X which is of size $T\times N\times K$, where $T=1500$, $N=1500$ and $K=10$. ...
2
votes
0answers
167 views

Procedure for variable selection + logistic regression when n is small, p is large, and data are unbalanced?

I have data that have been collected using case-control procedures, in which the population of positive cases is collected with a random sample of negative cases. This yields 62 positive cases and 179 ...
7
votes
4answers
2k views

Is there a way to use cross validation to do variable/feature selection in R?

I have a data set with about 70 variables that I'd like to cut down. What I'm looking to do is use CV to find most useful variables in the following fashion. 1) Randomly select say 20 variables. ...