Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [random-forest]

Random forest is a machine-learning method based on combining the outputs of many decision trees.

0
votes
0answers
8 views

How to get similarity matrix from a random forest model? [on hold]

I have trained a randomforestclassifier on a dataset with 652 samples. I have achieved 89.6% accuracy. Now I want to extract the similarity matrix from the trained classifer and do some clustering. ...
0
votes
0answers
12 views
0
votes
0answers
13 views

Coding Random Forrest in R [on hold]

I'm looking to code a random forest in R but am having a bit of trouble in my dataset. Before I get into the problem, let me reproduce my code below. The response variable of interest is '...
0
votes
1answer
22 views

Feature engineering with cross validation, then testing on a holdout data set?

We have 3000 samples for two classes, roughly 2000:1000. Our plan is to train a classifier on the samples but first to set aside 30% randomly selected stratified samples as a "holdout data set" for a ...
0
votes
0answers
38 views

R - multivariate random forest for variable importance

A little background - I'm trying to use multivariate random forest modeling to understand the importance of environmental variables on microbial communities (illumina sequencing) along a latitudinal ...
1
vote
1answer
32 views

Is it possible to have recall and precision of 0 while having an area under PR ~0.5?

As the title suggests, I am running a Random Forest classifier using Scala. To evaluate this classifier (and since I am handling highly imbalanced classes), I used the ...
4
votes
1answer
32 views

Top principal components versus most significant random forest variables

I was working on making a supervised learning model starting with a database of about 100 features and 1000 data entries. My goal is to predict a certain target variable. I tried three different ...
0
votes
0answers
10 views

Bootstrapping a statistics vs bootstrapping a model (e.g. random forest) [on hold]

Let y_s be the sample statistics and y_b as bootstrap mean of the statistics. When bootstrapping a statistics we use ...
3
votes
1answer
161 views

Using Random Forest variable importance for feature selection

I'm currently trying to convince my colleague that his method of doing feature selection is causing data leakage and I need help doing so. The method they are using is as follows: They first run a ...
0
votes
0answers
32 views
+50

%IncMSE random forests importance measure - why is mean prediction error divided by standard deviation?

Random forests have their variable importance calculated using one of two methods, of which permutation-based importance is considered better. In R's randomForest ...
2
votes
0answers
26 views

What value to impute for informative NA values in R without misleading model

I'm building a model (random forest) in R to predict a rare event (scoring a goal in soccer). I have event-level data, which provides a log of all the actions (pass, tackle, foul, save, shot, goal) ...
1
vote
0answers
22 views

Can a complex interaction term mean more than what it's composed of?

I'm cross-posting this question on both Economics and Cross Validated to get answers from a different perspective on each field. It is generally accepted to cross-post if the question is tailored to ...
0
votes
0answers
11 views

How can random forest variable importance of a pooled sample be greater than variable importance of individual samples?

Here's what I have: a data set with 30 different questions measuring customer experience (on a scale from 1-10), and a question measuring "Overall Customer Satisfaction" (Q1). The data set is ...
0
votes
1answer
37 views

High AUC but low R squared in a random forest classifier

I have been looking for an answer on this website and on Google but I can't seem to find a clear explanation anywhere. The problem is the following. I built a Random Forest model (using Python's ...
1
vote
1answer
20 views

Comparison Between Features for Random Forest or Decision Tree

In the random forest of sklearn package, will there be some relationships among features? For example: if value(feature_1) > value(feature_2); it is category A, else B. I read some materials online, ...
0
votes
0answers
14 views

WoE for Random Forest and SVM

There are a lot written about WoE (Weight of Evidence) transformation for the case of Logistic Regression Classifier. It works great. The question: can one (or does it make sense) to use this WoE ...
-1
votes
1answer
62 views

Normalizing input data for Machine Learning model ruins everything

I am trying to build a Random Forests model for regression purposes. I am using a very standard pipeline: a initialize the model (from Scikit-learn), I fit the model with ...
5
votes
1answer
49 views

Should OOB (Out Of Bag) error be less than a Test set error in Random Forests?

I am using the book, "An introduction to statistical learning with applications in R" and reading the section on using OOB to estimate the model error for Random Forests. The graph seems to suggest ...
1
vote
1answer
44 views

Monthly Times Series Modeling Approach

I have a machine learning problem and have been working in Sklearn/Pandas with Python to come up with an accurate model. I find myself deep in a rabbit hole trying to learn the best approach and how ...
1
vote
2answers
48 views

Why the performance of random forest is related to the order of training samples?

everyone! I find that the performance of random forest classifier in python seems to be related to the order of training samples. Can anyone help me to figure out the reason? Thanks very much! ...
0
votes
0answers
3 views

Using NDVI along with regular RGB and NIR bands for image classification

Is it plausible to use NDVI along with other regular bands for image classification related data processing? Recently, I came across a comment that RED and NIR band might interfere with NDVI or vice ...
1
vote
1answer
29 views

Is the out-of-bag (OOB) error of a random forest classifier overly optimistic if hyperparameters were learned via CV on the same dataset?

I am training a random forest classifier in a setting with such a low sample size that I cannot afford setting aside validation and test sets. I train the hyperparameters via cross-validation and ...
0
votes
0answers
19 views

Understanding Strength and Correlation in Random Forest

In the original paper of Random Forest, the author introduces a concept of strength and correlation. In the appendix section, a step by step guide to calculating these values are given for the ...
0
votes
0answers
17 views

Using decision tree + active learning for regression?

Existing literatures that concerns using decision tree to do regression is more limited compared to its classification companion. The same also holds for research regarding active learning. I am just ...
-1
votes
0answers
28 views

Can we perform ANOVA on random forest models?

I understand ANOVA can be done with glm and gam models. However, is it possible to do an ANOVA on a random forest model ? If so, what is the best way to proceed ? If not, what will be the altenative ...
0
votes
0answers
18 views

What is the meaning of “extremity of values for regression” when using sklearn.tree.export_graphviz?

I'm using sklearn.ensemble.RandomForestRegressor, and I would like to show the decision tree of one estimator using sklearn.tree.export_graphviz, but I don't understand what the meaning of extremity ...
1
vote
1answer
24 views

The importance() in randomForest returns different results, how to interpret this?

the importance has two variables %IncMSE and IncNodePurity, my results for these two are totally different...I'm predicting a player's value, and want to know which attributes are more important for ...
1
vote
2answers
30 views

Can a decision tree make a decision based on two variables at one split?

I know that the random forest algorithm works by generating a set of decision trees with a subset of features. Say I was using random forest as a classification algorithm looking at someone's data ...
-1
votes
0answers
41 views

Unbalanced classes multiclass

Certain ML algorithms have parameters which can be used to deal with the effects of unbalanced dependent variable classes. For example the random forest implementation in Sci-kit learn has the class ...
1
vote
3answers
51 views

LASSO or random forest (RF) to use for variable selection when having highly correlated features in a relatively small dataset with many features?

I have a cross sectional data-set with around 1000 features and 5000 observations. There are many features (no categorical features) which are highly correlated (higher than 0.85). I want to decrease ...
0
votes
0answers
33 views

What is the underlying reasoning behind sample.fraction or nSamp option in ranger and Rborist respectively?

I am trying to determine the application of the sample.fraction parameter and the nSamp parameter in the Ranger and Rborist R packages respectively, and what effect it can have on model training. In ...
1
vote
0answers
14 views

How to calculate OOB error vs. features plot

I have dataset (numeric values from 30 th number raster brick and some classes for every pixel. (class value was taken from RF classification result)). Well I know how to calculate importance of ...
0
votes
0answers
32 views

Doesn't high feature correlation decrease random forest accuracy?

I have generated a dataset of artificial data and want to distinguish two labels from each other using a random forest. I thought having correlated features in my dataset will decrease the algorithms ...
0
votes
0answers
14 views

Upper limit of number of independent features in a random forest?

I have just finished running a random forest algorithm and I am having trouble interpreting whether my results are correct. I ran this algorithm so I could get a measure of feature importance for ...
0
votes
1answer
32 views

How to do k-fold cross validation to get optimal specification in a random forest model?

i am an R beginner and i have to do a 5 or 10-fold cross validation in a random forest model. My problem is i have to do the cv manually and not with an package. What i want to do is: 1. Building k-...
3
votes
1answer
173 views

What makes a Random Forest random besides bootstrapping and random sampling of features?

After reading about random forests in the original paper and in textbooks I was under the impression that what makes the model random is bootstrapping - training each tree on a different random subset ...
2
votes
3answers
118 views

What are the advantages of Random Forest over Decision Trees

I am currently searching for the advantages of random forest over decision trees, but unfortunately I didn't find a research paper that does such a conclusion that summarize all the advantages of RF ...
0
votes
0answers
17 views

What does the number of features needed for machine learning rely on?

I generate datapoints in the form of two n-spheres with 25 features using the Monte Carlo Method. I want to feed the entire collection of points to an algorithm and let it separate the two clouds from ...
1
vote
1answer
22 views

Random forest feature importance with max_depth = 1

I am using sklearn to estimate a random forest classifier. Out of curiosity I have set max_features=None and max_depth=1. ...
0
votes
0answers
31 views

How to calculate bias and variance of Stacking model (a machine learning model)?

I'm doing bias-variance decomposition of Stacking model in regression case. The 1st-level learners include Random Forest, Extra-Trees and Gradient Boosting; then 2nd-level learner is Gradient Boosting....
0
votes
1answer
16 views

How does H2o handles missing values in DRF? [closed]

Just wanted to confirm that the h2o's implementation of RF (DRF) handles the missing values for both categorical and numerical features the same i.e., as a separate category?
0
votes
0answers
7 views

Multiclass classification- dealing with clusters within classes?

I'm currently dealing with a problem where I'm trying to predict how much a value x will change over time given input variables and am bucketing this change into separate classes (ie -100 to -50%, -50%...
0
votes
0answers
21 views

Imbalanced Dataset - Poor Evaluation

My dataset has about ~75,000 records with 39 features. Most of the features are categorical, so I have one-hot encoded them. About 14% are minority with label 0 and the rest 86% with label 1. I have ...
1
vote
0answers
26 views

Strange encoding for categorical features

I am reading through https://arxiv.org/pdf/1609.06676.pdf which presents an extension of the isolation forest algorithm so that categorical features may be taken into account. On page 5, the authors ...
0
votes
0answers
10 views

what does the DeltaCriterionDecisionSplit and NumPredictorSplit in CompactTreeBagger (Matlab)means?

when I try to use CompactTreeBagger in Matlab, I do not know what does the DeltaCriterionDecisionSplit and NumPredictorSplit means. Even though I search the offical guidence https://ww2.mathworks.cn/...
0
votes
0answers
21 views

Conditional Density Estimate Loss. Why the double integral?

I read RFCDE: Random Forests for Conditional Density Estimation. Just like it sounds, these folks trained random forests for making conditional density estimates. At inference time, density estimates ...
0
votes
1answer
15 views

Can we predict the monthly sales amount of the coming month without knowing the values of the independent variables of the coming month

I have a data set where the monthly sales of TMT bars and various other explanatory variables are present from April 2014-March 2018. I need to predict the monthly sales of the coming/next month. ...
1
vote
1answer
44 views

Sum of Random Forest prediction intervals?

I'm using a Random Forest model for prediction, where the value I'm interested in is the aggregate of these predictions (that is, the total sum of all predicted values). I want to also derive ...
3
votes
0answers
66 views

Overfitting in Random Forest Classifier?

I would like some help from you in a classification model that I am developing. In summary, the problem is: – Classification problem with binary outcome (0/1) – The classifier is a Random Forest ...
2
votes
1answer
38 views

Effect of continuous variable with many repeated values on Random Forest

I am trying to build a random forest regression model in R using all continuous panel data. There is a large amount of data relative to the number of predictors. Say 500,000 rows and about 40 ...