Tagged Questions

Random forest is a machine-learning classifier based on choosing random subsets of variables for each tree and using the most frequent tree output as the overall classification.

learn more… | top users | synonyms

0
votes
2answers
59 views

Analysis for checking if an Ensemble model is a better fit for a dataset than Primitive model

I have a dataset and have the option to apply either GLM (primitive) or a Random Forest (ensemble). So far the Random Forest is giving way better results than the GLM. As it is generally believed that ...
0
votes
0answers
29 views

Caret feature selection with customized random forest classifier

I'm following the Caret package tutorial for constructing customized functions for a recursive feature elimination. I can reproduce the provided example which is a random forest regression. However, ...
0
votes
0answers
25 views

interpreting y axis of a partial dependence plots

I have read through other topics on partial dependence plots and most of them are on how you actually plot them with different packages, not how you can accurately interpret them, So: I have been ...
0
votes
0answers
15 views

How to define samples in caret package?

I am using the caret package and need to train a random forest, where only certain samples should be in the held-out set. I want to define the sampling for each tree in the random forest, for say 100 ...
4
votes
1answer
34 views
+50

How to measure when error stabilizes (convergence) on Random Forests (or, when do I stop training)

I'm doing an implementation of Random Forests. As I was the original paper (page 11) and this nice book on the subject (15.3.1, page 592), they mention that when the out-of-bags error stabilizes (when ...
0
votes
0answers
18 views

Better calculation of ends of the curve when using R's RandomForest?

I've been using R's randomforest library to predict a series of (2000) products with prices ($10-$2000) and I'm noticing the low and high ends of the continuum are least accurate, especially when I ...
0
votes
2answers
48 views

Different results from several “passes” of Random Forest on same dataset

I've been playing around with the German Credit dataset available in Kuhn & Johnson's caret package for ...
-1
votes
2answers
41 views

Will RandomForest model work well if the correlation lies between the given attributes

Currently I am working on the data mining project and I am using RandomForest classification model for that. I have a few queries in that. Will the RandomForest handle if there is correlation ...
0
votes
2answers
83 views

random forest variables importance with continuous and categorical variables and unbalanced output

I am a bit lost in the literature regarding the random forest importance. I am aware that there are different methods. I have a binary output variables where elements labelled with 0 are much ...
0
votes
0answers
25 views

Accuracy low if test data belong to a single class

For my classification task I have two classes labeled 0 and 1. I am using Random Forests from sklearn package in python. I have two files for different classes. So I loaded the files, combined them ...
0
votes
0answers
10 views

randomForest sample size value formula

This question has been asked previously, but the answer didn't contain a formula - only a rule of thumb heuristic. Is there a formula or rule that exists for determining the appropriate sample size ...
0
votes
1answer
51 views

Random Forest confusion matrix

I've been creating some random forest models using the caret package in R. I don't have a large amount of data to work with so I'm using 10 x 10-fold CV in lieu of an independent test set. When I ...
0
votes
1answer
28 views

Does the randome forest for regression solution are interpretable and sparse?

I have regression problem scenario. basically I want to model a certian biological problem as regression models and the end my model should be interpretable. I need to have sparse model. so I'm ...
0
votes
0answers
38 views

Difference between tuneRF {randomForest} OOB error and Model OOB error

I have used tuneRF {randomForest} function to know best mtry and got OOB error is almost 19%, however when i run the model using randomForest am gettin OOB error 28%. I am running the model on same ...
0
votes
0answers
40 views

How can you print the decision tree of a RandomForestClassifier

Recently, I have noticed that there is a method sklearn.tree.export_graphviz documented here. However, I do not know how I can apply it to a ...
0
votes
0answers
20 views

R - randomForest - rfcv function - explanation in laymans terms

My name is Abhi and I am trying to teach myself data science by solving problems on the internet. In my current model I am using a random forest & the rfcv function to test the performance of the ...
0
votes
0answers
29 views

Random Forest - Huge Disparity between OOB Error and test data error

I am building my model in R and am using the randomForest package. My current model has 7 features and I see OOB error rate of about 14%. I also ran the rfcv in the random forest package to see how ...
1
vote
0answers
20 views

Using min leaf size above 1 in random forest/treebagging

Are there any advantages in using a min-leaf-size above 1 in classification with bagged trees(/random forests)? I would imagine that it could make the classification more robust as some area in the ...
0
votes
0answers
16 views

improving randomForest running time

This has already been asked and answered, but one of the answers didn't explain why a certain technique worked. So my question is "Why does calling randomForest(predictors, decision) instead of the ...
0
votes
1answer
34 views

Feature selection when bagging trees/random forest

I want to get a better understanding of feature selection and how the number of features affect performance when bagging trees. I am using Matlab's treebagger and I ...
2
votes
1answer
53 views

Predicting customer churn - train & test sets

I'm struggling with a problem where I'm trying to predict customer churn. I have monthly snapshot data going back several years, and tags for whether a customer left during a given month. My main ...
0
votes
0answers
35 views

Pitfalls of using random forest/GBM on proportion data?

I have a set of data with a dependent variable which represents a proportion, and many of the samples contain a response of 1. I would like to build a random forest or GBM regression model on the ...
3
votes
1answer
49 views

Is there a way to explain and generalise the decision made by random forest?

I saw a similar question was asked a few years ago, maybe there are updates on that. I would like to have a way to explain the decisions generated by random forest, possibly in a single tree. I ...
0
votes
1answer
35 views

Factor Analysis vs. Random Forest Feature importance

Could someone explain the intuition behind the difference of feature importance using Factor Analysis vs. Random Forest Feature importance. Does there lie an advantage in RF due to the fact that it ...
4
votes
0answers
22 views

Polling vs averaging in Random Forest models

Why is it that for Random Forest we take the average vote from each classifier in the ensemble rather than the average probability from each classifier in the ensemble? Is there theory behind why ...
1
vote
0answers
17 views

When is data slicing (and training separate models) worthwhile?

This question is more general about when one decides to train models on separate slices of a data set, rather than just one model on the entire training set altogether. So I'm working on a ...
0
votes
1answer
35 views

mtry tuning given by caret higher than the number of predictors

According to this discussion, it seems that the train function of the caret package returns a ...
0
votes
0answers
13 views

Dealing with numbers-based categorial data in rf regression: to standardize, or encode?

I'm working with the SEER cancer dataset, and I'm trying to use regression to calculate the months a breast cancer patient can expect to survive given certain variables. Some of these variables are ...
0
votes
0answers
19 views

Creating obligatory combinations of variables for drawing by random forest

Problem For my machine learning task, I create a set of predictors. Predictors come in "bundles" - multi-dimensional measurements (3 or 4 - dimensional in my case). The hole "bundle" makes sense ...
2
votes
1answer
25 views

With Random Forests, is it possible to provide an error term for each variable or each sample?

I'm wondering if there are any implementations of random forests that allow one to provide error terms for the inputs. An error term could be based on a per-variable basis, or on a ...
0
votes
0answers
27 views

Constructing Random Forests for binary classification by minimizing entropy

I'm looking to perform a binary classification using random forests, but I do not quite understand how to minimize the entropy of the data / what tests I should run on the nodes to do so. I'm fairly ...
1
vote
0answers
37 views

Why does Random Forest Regression perform worse than autoregression

I have a dataset of NFL games. Each game has one row for each team in the game. Each team's row contains the team's statistics in that game (such as points scored, passing yards, red zone attempts, ...
1
vote
1answer
59 views

Random Forest proof notation

I am having a bit of difficulty understanding the notation in equation (1) on page 4 of the following paper: https://escholarship.org/uc/item/35x3v9t4#page-4 Specifically, what do $E_{X,Y}$ and ...
1
vote
0answers
41 views

Random forest - proof of convergence

I'm having some trouble understanding Leo Breiman's proof that the generalization error of a random forest converges as the number of trees increases (here's a link to the paper). At Appendix I he ...
0
votes
0answers
16 views

Conditional Forest - Calculate Overfitting, OOB Error

My name is Abhi and I am trying to teach myself conditional forests. I have got a basic example working myModel <- cforest(Survived ~ .,data = ...
3
votes
0answers
67 views

Propensity score matching: using alternative methods to create a distance measure

I would like to use a greedy nearest neighbour method to do propensity score matching. Though I've little experience here, it seems that the distance measure used is generally a propensity score ...
1
vote
0answers
49 views

Random Forest for data imputation

Currently I am using Random Forest approach for Missing Values Imputation from missForest package in R. I faced the following problem: the algorithm works much longer than any other imputation ...
2
votes
1answer
47 views

R - ROCR Library - Understanding predict and prediction method

My name is Abhi and I am trying to understand the difference between predict and prediction. I am using the r language and my ide is rstudio. I have created a random forest model (r package ...
0
votes
2answers
52 views

Test data results does not match with cross validation results

I'm confused with my data I'm currently playing with. I have a data set which holds 58 attributes in 10000 instances. Attributes are 56 float values typically within 0 to 1. Then there is nominal ...
1
vote
1answer
85 views

Random Forest can't overfit?

I've read some literature that random forests can't overfit. While this sounds great, it seems too good to be true. Is it possible for rf's to overfit?
0
votes
2answers
62 views

Correct Evaluation of Random Forest on fixed training/test set

I have to test the performance of Random Forest on the same dataset (text classification) with about 118.000 instances of which about 1/3 is used for training and 2/3 is used for testing. The division ...
0
votes
0answers
28 views

How does one estimate the number of trees and the depth when using random forests?

I am using random forests for a class. I am predicting weight training. In scikit learn I always used a rule of thumb of a depth of 5, the depth x features rounded. I had 53 features, rounded to 50 ...
7
votes
0answers
91 views

Why does the scikit-learn bootstrap function resample the test set?

When using bootstrapping for model evaluation, I always thought the out-of-bag samples were directly used as a test set. However, this appears not to be the case for the scikit-learn bootstrap ...
0
votes
0answers
29 views

feature selection for longitudinal data

I have a longitudinal data which looks like this. Number of time points are different for each ID. Y is the binary response variable (take values 0 & 1) and X1-X20 are either continuous or ...
0
votes
0answers
90 views

Random Forest - understanding k fold cross validation

I am trying to improve my data science knowledge by solving problems available on the internet. I am currently using the R package randomForest to classify the ...
0
votes
1answer
115 views

R - Random Forest - Need help understanding the rfcv function

My name is Abhi. I am trying to teach myself data science by solving some of the problems available on the internet. My current data set has about 900 reccords & 10 features. I am trying to use ...
3
votes
2answers
75 views

Linear post-treatment of nonlinear regression

I have often found in practice, using nonlinear regression techniques such as feedforward neural nets or random forests, that the resulting actual-vs-fitted plot (on training set) seems obviously ...
3
votes
2answers
102 views

How does random Forest work for regression?

I am an absolute beginner in field of machine learning, I started doing titanic assignment in Kaggle and found(read some where) Random Forest is the best fit. I started reading about random forest and ...
1
vote
2answers
142 views

How to change threshold for classification in R randomForests?

All the Species Distribution Modelling literature suggests that when predicting the presence/absence of a species using a model that outputs probabilities (e.g., RandomForests), choice of the ...
1
vote
0answers
13 views

Does within-group heterogeneity negatively impact random forest classification?

I have two rather conceptual questions about random forest classifiers. Before we get there, I quickly want to lay out the problem I am working on: I have large a large data set consisting of 300 ...