Random forest is a machine-learning classifier based on choosing random subsets of variables for each tree and using the most frequent tree output as the overall classification.

learn more… | top users | synonyms

0
votes
1answer
11 views

Caret varImp for randomForest model

I'm having trouble understanding how the varImp function works for a randomForest model with the caret package. In the example ...
1
vote
0answers
31 views

Low explained variance in Random Forest (R randomForest)

I am using randomForest in R for regression, I have many categorical predictors (all of them have the same 3 categories (0,1,2)) and I want to see which of them can predict the response (continuous). ...
0
votes
0answers
5 views

Boruta score goes to minus infinity

I'm running the Boruta algorithm with a $179\times 36$ predictor matrix and a numerical response. Most of the variables have a score going to -Inf. Should I ...
0
votes
0answers
13 views

Justifying unsupervised clustering using Random Forest?

I have been looking at ways to carry out unsupervised clustering of data with both numeric and nominal (but not ordinal) variables. I also suspect non-linearity in the data. A possible solution would ...
0
votes
1answer
19 views

Confusion between caret randomForest predict() results and reported model performance

This question seems related, but the consensus was that the issue had to do scaling the data, which I do prior to training, so I don't think that's the issue: Issue on prediction with FinalModel of ...
0
votes
1answer
37 views

What is the equation for random forest?

I need an equation for random forest so that I can score fresh data I receive every week, based on beta estimates I got after building model using this ensemble methodology. Every week I do not want ...
0
votes
0answers
37 views

Random Forest: Important variables, Important values

I've been reading up on random forests and have come to a stumbling block in regard to their practical application (an probably fundamental understanding). This problem i in regard to important ...
1
vote
1answer
64 views

How do you validate your machine learning models?

I am wondering what approaches are commonly used for validating a classification or prediction models: Approaches that am using at the moment: Using truth-sets: - ROCs, Bootstrapping, Accuracy, ...
1
vote
0answers
40 views

R AUC never less than 0.5?

I'm doing some work with random forests in R using the randomForest package, and I've run into something that seems odd to me. Even when the data is completely ...
0
votes
0answers
11 views

Down-sampling with building models (specifically random forests)

I was wondering if anyone had ever used down-sampling to build random forests with data that has unbalanced classes. Basically down-sampling samples (with replacement) x*min from the population where ...
2
votes
1answer
29 views

Do random forest variable importance measures take into account the interactions?

Do random forest measures of variable importance (mean change of accuracy, mean change of Gini index) take the interactions into account? I think I know how we come up with the variable importance ...
0
votes
1answer
21 views

What's the best way to calculate survival time using outputs from random survival forest

I have built a random survival forest using R package randomForestSRC. The OOB error rate is around 10%. I was wondering whether anyone had some experience in utilizing the outputs from this model ...
0
votes
3answers
115 views

using random forest for missing data imputation in categorical variables ( in R)

I have following type of associated data. The following example step to generate associated variable. p number of variables and n is number of observations. ...
0
votes
1answer
33 views

randomSurvivalForest in R

I'm using the randomSurvivalForest package for R, version 3.6.4. I have been using it for a project for a while, with no problem. However, now I have added some additional predictors to my dataset, ...
2
votes
2answers
48 views

How to get rid of bias in data?

I have been trying to classify a set of data into one of four classes. The data has already been generated and I have set aside 10,000 for training and 2,000 for testing. I have also generated the ...
5
votes
2answers
59 views

How would one formally prove that the OOB error in random forest is unbiased?

I have read this statement many times but have never come across a proof. I would like to try to produce one myself but I'm not even sure on what notation to use. Can anyone help me with this?
2
votes
0answers
53 views

Why will a random forest not outperform a regression tree?

I have a training dataset with a binary response variable, 6 independent variables, and 21,000 observations. I've fit both an ordinary regression tree and a random forest (mtry = 2, ntree = 2000) and ...
1
vote
0answers
26 views

random forest and prediction

I am building a random forest model to make predictions. Response variable is treated as continuous but not really continuous, e.g., integers from 0 to 10. I have problems in constructing ...
0
votes
0answers
17 views

Is it possible to assign a different cost for each misclassification in a random forest?

I have a vector of weights for each instance which signifies the cost of each misclassification. How can I incorporate this while training a random forest?
1
vote
0answers
34 views

Unsupervised Random Forest for Visual Codebook generation

I'm trying to apply the bag of visual words approach to make scene classification. I started to use k-means to generate my codebook, but rapidly discovered its limitations. From one codebook ...
5
votes
1answer
90 views

Random Forest checklist

I am building a random forest model in R. Based on my research I (hope to) have come up with quite some understanding about how they work, and more importantly when they work. I simply would like to ...
3
votes
0answers
37 views

Random Forest - how to know if variables affect positively or negatively

I'm running a RandomForest in R on a set of data with many variables. Using varImpPlot() I know how important is each variable ...
1
vote
0answers
34 views

RandomForest: how to use a person_id and company_id attribute in machine learning

I'm trying to train a randomForest model in R in a 500k+ row dataset. So far so good, but now I'm trying to include factors person_id and company_id (non-unique) which both have a huge amount of ...
0
votes
1answer
41 views

Information gain with numerical data

I'm making a random forest classifier. In every tutorial, there is a very simple example of how to calculate entropy with Boolean attributes. In my problem I have attribute values that are calculated ...
0
votes
1answer
36 views

Random forest ML algorithm suitable for use on cluster based HPC?

I have developed a script using pythons scipy package to analyse a rather large model that I wish to solve, the model contains over 12gb of data, including over 500 parameters. Now running small ...
3
votes
0answers
55 views
+50

How can I include random effects into a randomForest

I'm not even sure that the question makes much sense, but I think I saw a couple of titles of papers where they proposed random forest with random effects. Is this possible in R?
1
vote
0answers
29 views

Random Forest regression model in R and data overfitting

I have trained my random forest model on a 74,000 training examples where each example consists of two proteins Amino Acids sequence (20 characters) and some numeric values representing the similarity ...
0
votes
0answers
9 views

Quality of NER classification decreases dramatically when sliding window transformation is performed

I'm writing a NER classificator now. That performs quite well even without window transformation (~80% F1-score for non-English language is quite well, AFAIK), but the strange thing is that when I ...
1
vote
1answer
59 views

Is random forest applied only to continuous response variable?

I am trying to apply random forest on a binary response variable, but it's saying the response variable has 5 or fewer unique values. Was it happening because random forest works only with the ...
0
votes
0answers
53 views

applying random forest in place of decision tree method in R

I am relatively new to random forest technique. I applied decision tree successfully to a data predicting 1/0 outcome. The predictors are mixture of continuous and categorical. I was trying similar ...
0
votes
0answers
23 views

Dropping predictor variables, based on variable of importance, effect of Random Forest Accuracy

I am trying to use Random Forest to accurately predict forested land cover classes using Landsat 7, climatic and geographical data. I have 23 predictor variables and 1 response variable. When I drop ...
0
votes
0answers
10 views

mtry values versus OOBE chart analysis

I was working on a dataset of 100,000 cases and 200 variables, and ran 7 different mtry values, trying to find the optimal one. Using the same data and same ntree values, I compared the different ...
4
votes
2answers
116 views

Feature selection with partial permutation

For feature selection, permutation tests are biased in favor of those categorical variables with a large number of levels [White1994]. Besides, it has been proposed [Deng2011] that partial ...
1
vote
1answer
60 views

Random Forest mtry Question

I am just looking to understand how mtry works in random forests. Please correct me if I am wrong. When you specify mtry (say 10), it takes 10 random variables from your data set and examines them ...
0
votes
0answers
43 views

Reduce Random Forest Model Size

I've created a regression model on my data using random forests in R. The output is quite large, I'm wondering if there's any way to reduce this to only the necessary pieces to make a prediction? The ...
1
vote
0answers
14 views

Analyzing the behavior of a non-linear model

I am working with a random forest model and would like to better understand how the variables impact the forest output. Using partial plots is useful, but this is basically a linear treatment of the ...
0
votes
0answers
32 views

How to explain the value of y-axis in Partial Dependency plot

sample numbers = n number of classes = 2 The function is $$F(x) = \sum_{i = 1}^nf(x,x_{ic}) $$ where $$f(x,x_{ic}) = \log p_1(x,x_{ic}) - \frac{(\log p_1( x,x_{ic} )+\log p_2( x,x_{ic}))}{2}$$ In ...
3
votes
0answers
28 views

Is it helpful to have monotonic features when using a random forest for classification?

I am training a random forest for binary classification. Here is a plot of one of my features, which is an integer giving the number of months since an event. The y-axis gives the proportion of cases ...
1
vote
0answers
38 views

Random Forest and substituting different variables

I am working on a model which predicts a binomial variable. I have millions of records and hundreds of variables to sample from. I have millions of records from individuals from each of the past ...
1
vote
0answers
43 views

Classification/Regression Tree with nonngeative response

I try to model the duration until a unit is inspected by a large number of possible explanatory variables. The duration is non-negative and the explanatory variables are factors and numerical ...
7
votes
5answers
230 views

How to perform imputation of values in very large number of data points?

I have a very large dataset and about 5% random values are missing. These variables are correlated with each other. The following example R dataset is just a toy example with dummy correlated data. ...
2
votes
2answers
105 views

random forest for large number of variables and predictions

I have very large number of variables compared to samples they are measured on. The following is example data in R. ...
0
votes
0answers
21 views

Response Range in Random Forest

I am using the random forest method (regression) to generate a predictor. Normally, I provide a response/answer along with each feature vector when I train my forest. However, for some of my ...
5
votes
2answers
90 views

Is it necessary to use cross-validatation to avoid overfitting when applying random forest algorithm?

Is it necessary to use cross-validatation to avoid overfitting when applying random forest algorithm? This is a question asked in my recent data scientist interview. Can anyone give any ideas? ...
0
votes
0answers
22 views

Optimal feature selection for MAPE criteria with RandomForest cross-validation

I am trying to optimize my set of features against random forest cross-validation using MAPE criteria. I tried forward selection with Univariate linear regression test (f_regression in sklearn), I ...
4
votes
3answers
254 views

Why doesn't Random Forest handle missing values in predictors?

What are theoretical reasons to not handle missing values? Gradient boosting machines, regression trees handle missing values. Why doesn't Random Forest do that?
2
votes
0answers
43 views

how is the “best” split at node determined in matlab RF package?

Does someone know how the "best" split at a node is determined in the MATLAB RF package? I have more than two classes, and just wondered how it is working. Is the package using the Gini impurity? ...
2
votes
1answer
65 views

Can I trust a model if I do not check the prediction error?

My main field is machine learning and 90 % of what I do is to try to improve the prediction error. Recently I have started to work with a medical group. They are mainly doctors so I do not know how ...
1
vote
3answers
110 views

Meaning of Bagged Random Forests?

I'm reading a paper that says that the authors used "bagged random forests". I couldn't understand this because as far as I know a random forest is a kind of bagging on its own. So a random forest is ...
2
votes
0answers
39 views

Discrete features are less important

I am trying to train a random forest classifier. As predictors, I keep both discrete features and continuous features (the discrete ones including booleans, counters, etc., and the continuous contains ...