Questions tagged [random-forest]

Random forest is a machine-learning method based on combining the outputs of many decision trees.

Filter by
Sorted by
Tagged with
146
votes
2answers
150k views

Gradient Boosting Tree vs Random Forest

Gradient tree boosting as proposed by Friedman uses decision trees as base learners. I'm wondering if we should make the base decision tree as complex as possible (fully grown) or simpler? Is there ...
139
votes
9answers
55k views

Obtaining knowledge from a random forest

Random forests are considered to be black boxes, but recently I was thinking what knowledge can be obtained from a random forest? The most obvious thing is the importance of the variables, in the ...
79
votes
3answers
61k views

Best way to present a random forest in a publication?

I am using the random forest algorithm as a robust classifier of two groups in a microarray study with 1000s of features. What is the best way to present the random forest so that there is enough ...
72
votes
2answers
40k views

Practical questions on tuning Random Forests

My questions are about Random Forests. The concept of this beautiful classifier is clear to me, but still there are a lot of practical usage questions. Unfortunately, I failed to find any practical ...
70
votes
3answers
134k views

How to actually plot a sample tree from randomForest::getTree()? [closed]

Anyone got library or code suggestions on how to actually plot a couple of sample trees from: getTree(rfobj, k, labelVar=TRUE) (Yes I know you're not supposed ...
63
votes
6answers
54k views

Do the predictions of a Random Forest model have a prediction interval?

If I run a randomForest model, I can then make predictions based on the model. Is there a way to get a prediction interval of each of the predictions such that I ...
56
votes
4answers
117k views

Random Forest - How to handle overfitting

I have a computer science background but am trying to teach myself data science by solving problems on the internet. I have been working on this problem for the last couple of weeks (approx 900 rows ...
56
votes
4answers
40k views

Difference between Random Forest and Extremely Randomized Trees

I understood that Random Forest and Extremely Randomized Trees differ in the sense that the splits of the trees in the Random Forest are deterministic whereas they are random in the case of an ...
55
votes
6answers
47k views

Is random forest a boosting algorithm?

Short definition of boosting: Can a set of weak learners create a single strong learner? A weak learner is defined to be a classifier which is only slightly correlated with the true ...
54
votes
4answers
79k views

Does the optimal number of trees in a random forest depend on the number of predictors?

Can someone explain why we need a large number of trees in random forest when the number of predictors is large? How can we determine the optimal number of trees?
53
votes
7answers
56k views

Why doesn't Random Forest handle missing values in predictors?

What are theoretical reasons to not handle missing values? Gradient boosting machines, regression trees handle missing values. Why doesn't Random Forest do that?
52
votes
3answers
22k views

Can a random forest be used for feature selection in multiple linear regression?

Since RF can handle non-linearity but can't provide coefficients, would it be wise to use random forest to gather the most important features and then plug those features into a multiple linear ...
52
votes
4answers
43k views

Random forest computing time in R

I am using the party package in R with 10,000 rows and 34 features, and some factor features have more than 300 levels. The computing time is too long. (It has taken 3 hours so far and it hasn't ...
49
votes
2answers
17k views

Random forest assumptions

I am kind of new to random forest so I am still struggling with some basic concepts. In linear regression, we assume independent observations, constant variance… What are the basic assumptions/...
45
votes
1answer
13k views

Do we have to tune the number of trees in a random forest?

Software implementations of random forest classifiers have a number of parameters to allow users to fine-tune the algorithm's behavior, including the number of trees $T$ in the forest. Is this a ...
45
votes
3answers
51k views

Won't highly-correlated variables in random forest distort accuracy and feature-selection?

In my understanding, highly correlated variables won't cause multi-collinearity issues in random forest model (Please correct me if I'm wrong). However, on the other way, if I have too many variables ...
45
votes
3answers
93k views

How to interpret Mean Decrease in Accuracy and Mean Decrease GINI in Random Forest models

I'm having some difficulty understanding how to interpret variable importance output from the Random Forest package. Mean decrease in accuracy is usually described as "the decrease in model accuracy ...
44
votes
5answers
18k views

Optimized implementations of the Random Forest algorithm

I have noticed that there are a few implementations of random forest such as ALGLIB, Waffles and some R packages like randomForest. Can anybody tell me whether ...
43
votes
2answers
60k views

Measures of variable importance in random forests

I've been playing around with random forests for regression and am having difficulty working out exactly what the two measures of importance mean, and how they should be interpreted. The ...
41
votes
1answer
18k views

Manually calculated $R^2$ doesn't match up with randomForest() $R^2$ for testing new data

I know this is a fairly specific R question, but I may be thinking about proportion variance explained, $R^2$, incorrectly. Here goes. I'm trying to use the ...
41
votes
3answers
86k views

How to interpret OOB and confusion matrix for random forest?

I got a an R script from someone to run a random forest model. I modified and run it with some employee data. We are trying to predict voluntary separations. Here is some additional info: this is a ...
40
votes
6answers
76k views

Improve classification with many categorical variables

I'm working on a dataset with 200,000+ samples and approximately 50 features per sample: 10 continuous variables and the other ~40 are categorical variables (countries, languages, scientific fields ...
39
votes
3answers
22k views

How are Random Forests not sensitive to outliers?

I've read in a few sources, including this one, that Random Forests are not sensitive to outliers (in the way that Logistic Regression and other ML methods are, for example). However, two pieces of ...
38
votes
3answers
22k views

Creating a “certainty score” from the votes in random forests?

I am looking to train a classifier that will discriminate between Type A and Type B objects with a reasonably large training set ...
36
votes
2answers
46k views

Relative importance of a set of predictors in a random forests classification in R

I'd like to determine the relative importance of sets of variables toward a randomForest classification model in R. The ...
30
votes
3answers
49k views

R: Random Forest throwing NaN/Inf in “foreign function call” error despite no NaN's in dataset [closed]

I'm using caret to run a cross validated random forest over a dataset. The Y variable is a factor. There are no NaN's, Inf's, or NA's in my dataset. However when running the random forest, I get <...
30
votes
3answers
8k views

How well does R scale to text classification tasks? [closed]

I am trying to get upto speed with R. I eventually want to use R libraries for doing text classification. I was just wondering what people's experiences are with regard to R's scalability when it ...
29
votes
7answers
22k views

When to avoid Random Forest?

Random forests are well known to perform fairly well on a variety of tasks and have been referred to as the leatherman of learning methods. Are there any types of problems or specific conditions in ...
29
votes
2answers
37k views

Is it essential to do normalization for SVM and Random Forest?

My features' every dimension has different range of value. I want to know if it is essential to normalize this dataset.
28
votes
5answers
14k views

How can I include random effects (or repeated measures) into a randomForest

I'm not even sure that the question makes much sense, but I think I saw a couple of titles of papers where they proposed random forest with random effects. Is this possible in R?
28
votes
1answer
16k views

Converting similarity matrix to (euclidean) distance matrix

In Random forest algorithm, Breiman (author) constructs similarity matrix as follows: Send all learning examples down each tree in the forest If two examples land in the same leaf increment ...
28
votes
1answer
31k views

Benefits of stratified vs random sampling for generating training data in classification

I would like to know if there are any/some advantages of using stratified sampling instead of random sampling, when splitting the original dataset into training and testing set for classification. ...
26
votes
3answers
39k views

What algorithms need feature scaling, beside from SVM?

I am working with many algorithms: RandomForest, DecisionTrees, NaiveBayes, SVM (kernel=linear and rbf), KNN, LDA and XGBoost. All of them were pretty fast except for SVM. That is when I got to know ...
26
votes
3answers
15k views

Feature importance with dummy variables

I am trying to understand how I can get the feature importance of a categorical variable that has been broken down into dummy variables. I am using scikit-learn which doesn't handle categorical ...
25
votes
2answers
10k views

Random forests for multivariate regression

I have a multi-output regression problem with $d_x$ input features and $d_y$ outputs. The outputs have a complex, non-linear correlation structure. I'd like to use random forests to do the regression....
24
votes
3answers
18k views

interpreting y axis of a partial dependence plots

I have read through other topics on partial dependence plots and most of them are on how you actually plot them with different packages, not how you can accurately interpret them, So: I have been ...
23
votes
11answers
20k views

Why is logistic regression called a machine learning algorithm?

If I understood correctly, in a machine learning algorithm, the model has to learn from its experience, i.e when the model gives the wrong prediction for the new cases, it must adapt to the new ...
23
votes
2answers
41k views

Random forest is overfitting?

I'm experimenting with random forests with scikit-learn and I'm getting great results of my training set, but relatively poor results on my test set... Here is the problem (inspired from poker) which ...
23
votes
4answers
39k views

Caret and randomForest number of trees [duplicate]

I am puzzled as to why the caret package in R does not allow tuning on the number of trees (ntree) in a random forest (specifically in the randomForest package)? I cant imagine this is an oversight on ...
23
votes
4answers
11k views

Is there a Random Forest implementation that works well with very sparse data?

Is there an R random forest implementation that works well with very sparse data? I have thousands or millions of boolean input variables, but only hundreds or so will be TRUE for any given example. ...
23
votes
5answers
15k views

How to control the cost of misclassification in Random Forests?

Is it possible to control the cost of misclassification in the R package randomForest? In my own work false negatives (e.g., missing in error that a person may have a disease) are far more costly ...
22
votes
5answers
46k views

R's randomForest can not handle more than 32 levels. What is workaround?

R's randomForest package can not handle factor with more than 32 levels. When it is given more than 32 levels, it emits an error message: Can not handle categorical predictors with more than 32 ...
22
votes
2answers
30k views

What does “node size” refer to in the Random Forest?

I do not understand exactly what is meant by node size. I know what a decision node is, but not what node size is.
21
votes
5answers
23k views

Random forest vs regression

I ran an OLS regression model on data set with 5 independent variables. The independent variables and dependent variable are both continuous and are linearly related. The R Square is about 99.3%. But ...
21
votes
4answers
3k views

“Semi supervised learning” - is this overfitting?

I was reading the report of the winning solution of a Kaggle competition (Malware Classification). The report can be found in this forum post. The problem was a classification problem (nine classes, ...
20
votes
2answers
4k views

How does random forest generate the random forest

I am not an expert of random forest but I clearly understand that the key issue with random forest is the (random) tree generation. Can you explain me how the trees are generated? (i.e. What is the ...
20
votes
3answers
11k views

Combining machine learning models

I'm kind of new to datamining/machine learning/etc. and have been reading about a couple ways to combine multiple models and runs of the same model to improve predictions. My impression from ...
20
votes
1answer
21k views

Is R-squared value appropriate for comparing models?

I'm trying to identify the best model to predict the prices of automobiles, using the prices and features available on automobile classified advertisement sites. For this I used a couple of models ...
19
votes
4answers
2k views

Is random forest for regression a 'true' regression?

Random forests are used for regression. However, from what I understand, they assign an average target value at each leaf. Since there are only limited leaves in each tree, there are only specific ...
19
votes
3answers
54k views

How does `predict.randomForest` estimate class probabilities?

How does randomForest package estimate class probabilities when I use predict(model, data, type = "prob")? I was using ...

1
2 3 4 5
42