Random forest is a machine-learning classifier based on choosing random subsets of variables for each tree and using the most frequent tree output as the overall classification.

learn more… | top users | synonyms

1
vote
2answers
31 views

what does the correlation of Random forest regression tool in R represent

I've built a random forest model (regression model) using randomForest package in R, and I calculate the correlation between the predicted values and the actual ones in order to know how the trained ...
0
votes
0answers
12 views

Convergence of Estimation

The issue is, we can have a bunch of regression models to do prediction for the test set. For a specific testing data, some perform good and some do bad. How can I use existing method or design a ...
2
votes
2answers
70 views

odds ratio from decision tree and random forest

I am using a decision tree and random forest for a classification problem. The output is binary {0,1} and some of the input variables are categorical while the others are continuous. I would like ...
0
votes
1answer
22 views

Should I specify Prior or Cost matrix with Tree Bagger in Matlab

I'm trying to create Random Forests in Matlab and there are more observations in some classes than there are in others. Do I need to specify this as a cost matrix or as a prior probability or will ...
5
votes
0answers
100 views

Random forest on multi-level/hierarchical-structured data

I am quite new to machine learning, CART-techniques and the like, and I hope my naivete isn't too obvious. How does Random Forest handle multi-level/hierarchical data structures (for example when ...
3
votes
1answer
58 views

Unsupervised Random Forest using Weka

I am having some issues understanding how unsupervised Random Forest works according to Breiman. I only have unlabeled data, so the thought arose to use unsupervised Random Forest and use the ...
2
votes
1answer
77 views

Relative importance of a set of predictors in a random forests classification in R

I'd like to determine the relative importance of sets of variables toward a randomForest classification model in R. The ...
0
votes
0answers
48 views

How do I get coefficients of a random forest model?

I am using randomForest to generate a model, and at the end I don't know how I can get the final coefficients that the model is fitting. I know that for linear ...
1
vote
0answers
33 views

R: What do I see in partial dependence plots of gbm and RandomForest?

Actually, I thought I had understood what one can show a with partial dependence plot, but using a very simple hypothetical example, I got rather puzzled. In the following chunk of code I generate ...
3
votes
1answer
48 views

Finding interactions using randomForest

I am trying to use randomForest in R to find interaction terms to add to a model. My plan was to fit trees with maxnodes=4 (two ...
2
votes
1answer
48 views

Imbalanced training dataset and Random Forest regression model

I have a large dataset (>300,000 observations) that represent the distance (RMSD) between proteins. I'm building a regression model (Random Forest) that is supposed to predict the distance between any ...
1
vote
0answers
13 views

Gap in testing random forest using samples with different construction

I'm using a random forest to predict a true/false classification. I have roughly 20,000 registers per month over a year. If I leave out of my train set a random 20% of the data, I get ~40% KS. If ...
3
votes
2answers
54 views

How do I know which class my random forest model is predicting?

I have a random forests model with which I am trying to predict species presence or absence. This is my code: ...
1
vote
1answer
73 views

Confusion matrix calculation in random forest classifier in R

After training using random forests on the iris dataset, I get an OOB error and a confusion matrix. ...
3
votes
1answer
53 views

Understanding random forest, gini, and KS

I'm a beginner machine learning user, doing my first predictive model using random forest. I have some questions regarding the way to measure how good a model is (Gini area from roc curve, and KS), ...
2
votes
1answer
48 views

unbalanced samples random Forests

I am trying to predict species presence or absence using randomForest in R (classification). In fact, I am trying to do it for several species, in separate models. For a couple of the species, the ...
0
votes
0answers
39 views

Random forest highly imbalanced dataset: how to test _ create ROC curve?

I have a dataset containing 10,000 examples with 8 features. I would like to create a random forest to classify this dataset in two classes, more specific: substrate / no substrate. In this dataset ...
1
vote
1answer
61 views

building a classification model for strictly binary data

i have a data set that is strictly binary. each variable's set of values is in the domain: true, false. the "special" property of this data set is that an overwhelming majority of the values are ...
0
votes
0answers
27 views

Random forest: confounding factors

I have N variables in K samples. There is a classification variable, T (treatment), and a confounding variable -- sex. Unfortunately, in the "no treatment" (CTRL) group there are significantly more ...
1
vote
0answers
55 views

Predicting and calculating test mse using cforest R

I'm new to the cforest package and am trying to create a cforest model to predict a new test set and calculate the model test MSE. My data is split into d.train ...
3
votes
1answer
43 views

Why the trees generated via bagging are identically distributed?

I have problem in intuitive understanding of following arguement: "The trees generated via bagging are identically distributed, thus the expectation of the average of a set of trees is the same as ...
1
vote
1answer
78 views

Interpret Variable Importance (varImp) for Factor Variables

When I run variable importance on a random forest (or any other model), the factor/categorical variable names have the factor name as the suffix. For example, ...
1
vote
0answers
20 views

Predicting a Certain Type of failure and deciding the input time series

I am trying to predict the time to certain type of failure given the following data on Certain Factory Equipments. The data I have are readings collected every day for sensor installed on those ...
0
votes
0answers
49 views

Interpreting RandomForest variable importance in case of multiple polytomous dummy variables

I have some trouble interpreting R's RandomForest variable importance measures when using multiple polytomous dummy variables. Take the following example: I have paired country data, e.g. ...
0
votes
2answers
84 views

Random Forest Regression Overfitting - Quantile Test on Test Data

I have fit a random forest regression model to training data (used 65% of data for training). The data has approximately 40,000 observations and 100 features. I fit a random forest regression in R ...
3
votes
1answer
106 views

Trend Analysis of feature importance over time in R

I'm running an experiment on a Streaming Classification Model (an Online Random Forest) that I've created. If that is a completely foreign concept to you here is a presentation I did on it recently: ...
1
vote
0answers
19 views

Very Naive Questions about training group selection in Random Forest or Tree Based Method

I am just beginning to discover the possibilities of machine learning and I have a naive question: First, does Recursive classification tree method (as shown in party package in R, the ctree ...
2
votes
1answer
102 views

Monte Carlo simulation vs. machine learning algorithms: what is the difference in application?

I have been doing some research on different type of machine learning (ML) algorithms such as random forest/SVM etc. in order to model and best predict pharmaceutical needs of patients suffering from ...
1
vote
0answers
52 views

Random forest performs worse than single CART tree?

I have a data sample of ~5000 observations, ~700 predictors and 2 classes. I've built a classification model based on RF with 500 trees using randomForest R library. Than I've estimated the ...
1
vote
0answers
41 views

Random forest for binomial data

I have data where the response is of the form (k,n) - k successes, n trials for a given subject. The covariates are just some subject-specific predictors. Is it possible to use randomForest for ...
2
votes
0answers
53 views

Dissimilarity with earlier features part of cost function

I am using a RandomForest on features (pixels) of images, and I am considering adding cost for "similarity to already other included features" to the cost function. Imagine you have a current RF ...
1
vote
0answers
47 views

Tricks for a very fast implementation of Random Forest

I am implementing my own Random (regression) Forest algorithm and am looking for tricks to speed up the estimation of forests on large datasets. So far I have implemented three main tricks: 1) Use a ...
2
votes
1answer
57 views

Are random forests Bayesian?

Hopefully my naivety doesn't shine through in this question: Are random forests Bayesian?
2
votes
0answers
30 views

Identifying what weights to give to each class in a Random Forest

I am using a randomForest package in R to discriminate between 4 categories. My data consists of 80+ observations and is heavily unbalanced with around 70% of all observations being in a single ...
1
vote
1answer
70 views

How to model features with NULL (not necessarily missing) values

I’d like some advice on my options for modeling features (to be used in R) in the following scenario: I want to make a binary prediction using the positions of the planets as the predictors (the X’s). ...
1
vote
0answers
21 views

Random classification forests for extremely sparse response variables

I have a response variable that can be $A,B,C$. It is very sparse, meaning 99% of the sample is $B$ and the rest is approximately evenly divided between $A$ and $C$. How do I predict this variable in ...
0
votes
0answers
17 views

Early split decision criteria for fast random (regression) forest estimation

Suppose I am on a node in a $regression$ tree and I am using running estimates of $\sum_{i \in Region_1} (y_i - mean(y_i)_{Region1})^2$ (and the same for Region 2) to determine whether to split the ...
1
vote
0answers
13 views

How to get more continuity in regression forest output

I am using a regression forest. What I have noticed when I plot the quantile distribution of the forest's output is that over a long stretch of quantiles (e.g. $\tau \in [0.1,0.3]$), the output will ...
3
votes
1answer
49 views

How to split nodes in regression trees

I am looking for a comparison of different regression tree node splitting approaches within the random forest framework. I am looking at the trade-off between ensemble accuracy/reliability (holding ...
0
votes
0answers
51 views

what to do with 0.5 class probabilities ?

I am currently training a random forest regressor (scikit learn) on the Titanic dataset. My question is related to this issue ...
0
votes
0answers
16 views

Discretization of the split in random forest estimation

If the dataset I'm working with has too many independent variable dimensions, I may need to choose a proper subset of all possible locations in my feature vector to determine whether I am going to use ...
2
votes
1answer
76 views

Difference between Random Forest and MART

I just wonder what is the difference between Random Forests and MART (Multiple additive regression trees)? I have read a few articles, e.g.: [1] L. Breiman. Random Forests. Machine Learning 45 ...
0
votes
0answers
34 views

Calculating the oob strength and correlation estimates for a regression random forest

I'm currently working on a regression application of random forests. I'd like to produce strength and correlation graphs of the tree ensemble with varying number of random features selected (mtry in ...
1
vote
1answer
65 views

How to incorporate constraints in random forest output

Suppose I am doing random forest classification of labels $A$,$B$,$C$,$D$. There is some theoretical ordering to this output such that when $A$ is more likely than $B$, $B$ is also more likely than ...
0
votes
1answer
48 views

Can we remove trees from a random forest with poor OOB error to improve generalisation?

My objective is to improve out of sample generalization of my random forest while holding the number of trees constant. Suppose that I am only allowed to use $n$ trees on the out of sample data but ...
2
votes
0answers
58 views

Probability extraction from random forest classifier

I have a random forest to perform classification. I need the real probability of the predicted class. They take a feature vector X and output a predicted class C. Additionally we can compute the ...
1
vote
1answer
67 views

Does RandomForest ignore spatial independence?

I have 5 variables for each countries of the world and I need to analyze their effect and interactions on an independent variable. Random Forest would be adequate for my scope as it deals with ...
1
vote
1answer
44 views

Weird bootstrap bias for Predictor Importance (MeanDecreaseAccuracy) in Random Forests

Below, using R, I: 1. Create a data set with a bunch of factors. All of them are predictors and 'y' is the dependent variable. 2. I run a classification Random Forests for y with predictor importance. ...
3
votes
0answers
49 views

What is the purpose of working on a logit scale in partial dependence plots?

What is the purpose of working on a logit scale in partial dependence plots (in binary classification)? One could simply go about as follows: Grow a forest Suppose ...
0
votes
0answers
20 views

Neural Net on results from Partition Bootstrap Forest - validity?

I am seeing some promising results with this method for my 'nutrition' dataset but am wary about modelling on results of a model. Problenms with this data I cannot seem to reduce my many correlated ...