Random forest is a machine-learning classifier based on choosing random subsets of variables for each tree and using the most frequent tree output as the overall classification.

learn more… | top users | synonyms

1
vote
0answers
14 views

Random Forests overfitting/unbalanced classes?

Suppose I am using random forests where the classes are highly unbalanced. How do you detect over fitting and what can you do to avoid it? Breiman says in his paper that random forests do not overfit, ...
0
votes
0answers
12 views

Resample random forest OOB to choose number of trees?

My post was inspired by this one (http://stackoverflow.com/questions/29290916/scikit-learn-random-forest-classifier-how-to-produce-a-plot-of-oob-error-agains) Although random forest models do not ...
1
vote
1answer
19 views

Different probability values when using DecisionTreeClassifier and RandomForestClassifier

I'm studying the Random Forests and I made a little example to validate my knowledge. I create two classifiers, one with the DecisionTreeClassifier and another with RandomForestClassifier. After I ...
3
votes
1answer
46 views

Effect of categorical interaction terms with random forest machine learning algorithm

Thanks in advance for the help. I have moderately large dataset (around 7000 samples) with numerous categorical predictors and a single binary response. All of the predictors are categorical. ...
0
votes
0answers
22 views

caret: using random forest and include cross-validation

I used the caret package to train a random forest, including repeated cross-validation. I’d like to know whether the OOB, as in the original RF by Breiman, is used or whether this is replaced by the ...
0
votes
1answer
40 views

Machine learning framework for SVM, Random Forest

I need an library, or something that is already done for SVM and Random Forest algorithms. Can you give me some ideas? I don't have experience and I don't know what to choose. The restriction of my ...
0
votes
0answers
6 views

Classifier with interchangeable features

I have a situation in which the features used in a classifier are multiple instances of the same kind of measurement, in random (or unknown) order; thus, a sample x1, x2, ... xn -> classA could with ...
3
votes
1answer
57 views

Plotting learning curves for any classification algorithm

As recommended by Andrew Ng in his great course on machine learning, I would like to plot the learning curves for experiments I am running with Random Forest and SVM algorithms. The learning curves ...
0
votes
0answers
11 views

feature slection in random forest in python

I have a dataset consisting of 24 numeric features and about 7000 rows, i am applying random forest to get the binary classification, So please tell me how to find only the relevant features to get ...
1
vote
0answers
17 views

Best Random Forest model converging to bagging: What does it mean? (R)

I am performing a grid search to tune the Random Forest parameters m and nodesize. I have 79 variables, and the best model, in terms of OOB error, is a model with 76 variables (OOB error = 0.137). So, ...
0
votes
0answers
69 views

Random Forest accuracy 0.98, is it too much?

I am using about 256 predictors and target is sales. I am using a software called Alteryx which is R based. I have tried to run Random Forest, Spline model and Neural nets on same data. I used ...
0
votes
0answers
25 views

Random Forests: duplication and stratification when dealing with imbalanced data

From here, here, and here, it seems that one option when applying Random Forests on imbalanced data sets is duplication followed by stratification. I definitely see the benefit of oversampling rare ...
4
votes
0answers
29 views

Does Breiman's random forest use information gain or Gini index?

I would like to know if Breiman's random forest (random forest in R randomForest package) uses as a splitting criterion (criterion for attribute selection) information gain or Gini index? I tried to ...
2
votes
1answer
46 views

Negative $R^2$ at random regression forest [duplicate]

I am currently writing my master's thesis about random forests and just started to work with the R software. When I am running my model the output looks like this: ...
1
vote
1answer
35 views

R randomForest R replace=TRUE pro's and con's

When using R randomForest package I use replace=TRUE, which then dictates to: if (replace) nrow(x) else ceiling(.632*nrow(x)) I was wondering if anyone knows of ...
4
votes
1answer
104 views

Should you tune 'ntree' in the Random Forest algorithm?

In the original paper, I was under the impression that the RF couldn't really overfit. However, in practice I'm seeing that increasing 'ntree' sometimes increases test set error. Is this due to ...
0
votes
1answer
29 views

Applying randomforest algorithm (fit) on new data without recomputing the fit [closed]

I have a need to do realtime predictions for individual rows of data based on a previously computed randomForest algorithm. How can I run the "predict" command without recomputing "fit" on the entire ...
2
votes
2answers
49 views

Classification & Regression Trees (CART)

It seems that most CART examples I encounter involve splitting the sample into a training sample and test sample. This split sample method strikes me as terribly inefficient unless you have a large ...
2
votes
0answers
41 views

How to increase the performance of random forest classifier?

I have a text classification task. These are the metrics for different languages at present: class1: 0.6823 class2: 0.7450 class3: 0.66 class4: 0.6719 How can I ...
0
votes
0answers
27 views

Get bagging sets of random forest in scikit-learn

In scikit-learn each tree of a random forest is trained with a set of samples drawn with replacement from the training set. To do some analysis of a trained forest, it would be nice to know, which ...
2
votes
1answer
32 views

The absolute value of importance is not very helpful, than how to calculate the relative importance of variables in a random forests Regression [duplicate]

I have used a random forest model for regression analysis. Now, I am having difficulty in working out what can be used in measuring variable importance. The importance function provides the mean ...
1
vote
1answer
47 views

Random forest regression on a given interval

I'm training a random forest regression model on a dataset that consinsts of values in the range of 0-50. It has many values close to zero and only 500 observations. The R^2 is also small, about ...
0
votes
1answer
35 views

prototypes and replacing missing variables in Random Forests algorithm

I'm new with Random forests Classification algorithm, and I have some questions about concepts confused me, What is the role of prototypes in the classification operation, Are they the core of the ...
4
votes
1answer
57 views

Highly correlated variables in random forest

In my understanding, highly correlated variables won't cause multi-collinearity issues in random forest model (Please correct me if I'm wrong). However, on the other way, if I have too many variables ...
0
votes
0answers
44 views

Different results from randomForest via caret and the basic randomForest package

I am a bit confused: How can the results of a trained Model via caret differ from the model in the original package? I read Issue on prediction with FinalModel of RandomForest in R using the CARET ...
2
votes
1answer
45 views

Unsupervised Clustering using randomForest

Outline of clustering technique using Random Forest A synthetic data is created by randomly sampling from the data of interest. It is used as the base line to measure the "structureness" or ...
2
votes
1answer
41 views

Classification when some classes are dependent

I think my problem can easier be explained via an example: Assume we have a dataset containing the images of 10 different mammals, let's say lion, elephant, cat, ... and horse. We have a 20-class ...
2
votes
1answer
85 views

How to document a Random Forest result (final model)?

Using Random Forest to predict dichotomous variables (for classification) I encountered the problem how to best document this model, i.e. I want the user to reproduce/use the final model I created on ...
0
votes
0answers
34 views

RandomForest Classifier With Very High Success Rate

I'm having a weird problem that may suprise you all. My classification rate is too high on my test set. I'm using scikit-learn packages, and I'm very suspicious of these classification rates, as they ...
2
votes
1answer
56 views

partial plots from a Random forest classifier for binary predictors

I'm working on a classification problem with continous and categorical predictors with Random Forests (RF). I'm particularly interested on RF as we avoid the specification of the functional form. ...
3
votes
1answer
60 views

Should I use Mean Square Error or Classification Rate?

I am a self-taught person and I would like your help. I am learning about predictive modeling in general, and I'm also trying to do predictive modeling for a specific problem. I am exploring ...
0
votes
0answers
33 views

Correlation between decision trees

How do you find the correlation between decision trees? I know this is an issue when working with random forests, but I can't find an explicit formula anywhere. I only get that random forest reduces ...
0
votes
0answers
27 views

Incorporating systematic error in (spatial) predictive modelling

I have created a model (random forest) and withheld 20%. When I apply the model to the withheld dataset and check the residuals against the real values I can see there is a systematic error e.g lower ...
4
votes
2answers
102 views

Can I safely use variable importance of a random forest in a paper?

Background: I just started with machine learning and I'm considering using it on old data based on which I'm writing a paper. The paper deals with radiation-induced lung damage and the data comprise ...
0
votes
0answers
27 views

Should I normalise (to sum to 100%) the MSE reduction for variable importance in a random forest?

I am investigating the importance of a set of variables in a linear model. I am conducting a random forest analysis and using the permutation-based Mean Square Error (MSE) reduction as a measure of ...
0
votes
0answers
30 views

Usage of standard deviation in random forests regression (for expected improvement)

It seems to me that the awareness of this problem is not high enough. Often the standard deviation (so the sd from each prediction of each tree in the forest) from the random forest regression is ...
0
votes
0answers
90 views

In recursive feature elimination in random forest, why are all features selected?

I am trying to use the recursive feature elimination in caret package. Here's the code; ...
0
votes
0answers
12 views

Adapt sampling in Random Forest (R)

I have a number of objects ($a,b,c,...$) which are observed over time. From each object there may be multiple observations (e.g. $a_1, a_2, a_3, b_1, b_2, c_1, ...$) with a numeric outcome. Note that ...
1
vote
0answers
16 views

Whats is meant by proximity in random forests?

I came across the term proximity in random forest. But I couldnt understand what it does in random forest. How does it help in classification??
0
votes
0answers
31 views

Understanding the (multiple) OOB columns in R's randomForest output

This is somewhat related to this post from 2012, however I want to focus specifically on the metrics generated while the randomForest model is running and make sure ...
0
votes
0answers
54 views

What do these decision boundaries indicate in random forest and svm?

I was working on data science harvard homework problem. It is a two class classification problem in which they plot the decision boundary for random forest, svm and decision tree. The problem has 2 ...
0
votes
1answer
53 views

determining how “important” a feature is in predicting a target in decision trees

Random forests allow us to compute a heuristic for determining how "important" a feature is in predicting a target. This heuristic measures the change in prediction accuracy if we take a given ...
0
votes
0answers
38 views

what is the difference between bagging and boosting in random forest?

I understand what is bagging and how it is applied to random forest. But how is bagging different from boosting. If boosting is different from bagging, how can boosting be applied to random forest?
0
votes
2answers
52 views

Predictive Model - Increase Pediction Accuracy for Less Likely Events

I am trying to build a model that predicts the which binary category a respondent belongs to (0 or 1). I have demographic variables (all categorical) and a few 10 point questions. I have built a few ...
1
vote
1answer
63 views

How does caret handle factors?

I have been testing conditional trees and random forests with caret, and I've noticed it does something weird with factors. So, for example, a ctree using the base dataset ...
0
votes
0answers
12 views

Why is MeanDecreaseGini over 1 in RandomForest package in R? [duplicate]

I am using R package randomForest, and calculated MeanDecreaseGini as below. ...
0
votes
0answers
55 views

How to do ranking with scikit-learn random forest model

I have a training dataset that I've developed, that has the following format: ------------------------------ | User ID | Item | Label | ------------------------------ | 001 | umbrella | 0 ...
0
votes
0answers
74 views

Deep Learning Predictions Seem Off

I'm doing a linear regression using the h2o deep learning interface with R. I'm comparing the predictions to the ones I'm getting from the randomForest R module. The predictions from randomForest ...
1
vote
1answer
33 views

using training data in final model output

I have customer data for around 400,000 customers where 270,000 of them are current customers and 130,000 of them are past customers who churned, what I am doing is classifying them as 0 (non-churn) ...
1
vote
0answers
54 views

Machine learning/random forests with noisy response data

Machine learning techniques like random forests seem to assume that the responses in the training set are known perfectly. Specifically for regression applications, it seems one needs to account for ...