Questions tagged [random-forest]

Random forest is a machine-learning method based on combining the outputs of many decision trees.

Filter by
Sorted by
Tagged with
1
vote
0answers
8 views

Can RFs find a product interaction between two independent variables?

I'm doing the FastAI course on ML, and the main topic that is currently being discussed is random forests. Jeremy Howard explains how random forests, unlike something such as logistic regression, can ...
0
votes
0answers
8 views

Is there a rule of thumb when comparing model accuracy of training and testing set?

When we compare our model accuracy on our training and testing data, a large difference is good indicator that our model might be overfitting. But how large must this difference be? Is there any rule ...
0
votes
0answers
9 views

How can i apply Random Forest regression in time series data [duplicate]

I have daily water level data of 1990 to 2010 with precipitation,solar,temperature humidity and wind data.I want to apply random forest method in this time series data for estimating water level.But ...
0
votes
1answer
19 views

Potential for basic inference from comparison of cross-validation and testing scores using a Random Forest Regressor

I have run a series of Random Forest Regression models using different feature combinations (F-x). These feature variables are geological features. As some of these features are different ways to ...
1
vote
0answers
24 views

Order of features for gridsearch and model fitting

Assuming that the same columns (i.e., features) are used for hyperparameter tuning and model fitting, and ensemble models are used for modeling (e.g., Random forest or XGboost), then does the order of ...
0
votes
0answers
12 views

The most basic question about Feature Important or Permutation_Importance

Consider the XOR gate with three inputs. The truth table will be: Now all the variables on their own are near random as far as the model is concerned. Each input 1 or 0 has a 50% chance of being ...
1
vote
0answers
12 views

Find correlations using random forest [closed]

Having data in this format: ...
0
votes
1answer
14 views

What is the default node splitting process carried by sci-kit's RandomForestRegressor when all features and target are continuous?

I have some data containing several features, mainly continuous variables. Implementing the randomForestRegressor algorithm from the sci-kit package in Python is relatively simple and results look OK....
1
vote
1answer
36 views

Random Forest pruning vs stopping criteria

I have recently noticed that SciKit-Learn now supports Cost Complexity Pruning, which is great. Since this has been implemented, should I still use other regression trees/ random forest hyper-...
0
votes
0answers
19 views

R - Making legit RandomForest results reproducible with set.seed

I guess that my question is kinda weird but: I'm working on a university project where I have to use a RandomForest model to predict if patients have depressive tendencies. And while I'm getting ...
1
vote
1answer
47 views

Algorithm selection rationale (Random Forest vs Logistic Regression vs SVM)

I want to understand the criteria of selection of ML algorithms i.e what are the guidelines on which algorithm to be selected in which case ? The reasons I know are : Logistic regression to be ...
0
votes
0answers
35 views

Multivariate longitudinal classification using Random Forest Classifier

As the title suggests, I have a multivariate longitudinal dataset (also called panel data). (I have over 100.000 observations. The time period is X years. This means that for every year I have the ...
1
vote
2answers
56 views

Random Forest in R - How to perform feature extraction and reach the best Accuracy result?

I'm working on a university project where I need to build a Random Forest model in R to predict if patients have depressive tendencies according to their EEG-data. I already preprocessed the data and ...
0
votes
1answer
36 views

How do I split the data using repeated cv?

I am working on a coursework and I have the following task: Formation of training and test sets in R using the methods below: • Repeated CV for Bagging type classifier • Repeated CV for Stacking ...
0
votes
1answer
19 views

Selecting Feature weights

I use the knn Classifier for a binary classification problem. To improve the classification results I would like to multiply features by weights that are learned from data. I found different ways to ...
1
vote
0answers
11 views

Regression trees and choosing threshold values to minimize mean squared error

I'm trying to learn a decision tree for a regression problem. Each node of the generated tree, will split by the criterion variable < threshold for some ...
1
vote
0answers
21 views

Need Help in interpretation of results for random forest model

I was working on a binary classification problem where the ratio of Y(Unprofitable)/N(Profitable) is 52/148, in a train set with a sample size of 200. I got expected results with random forest model ...
0
votes
0answers
19 views

Taking temporal coherence into account: HMM

I would like to detect sleep stages in 30s intervals, given 4 EEG and 1 EMG signals. Since my EEG and EMG data are just timeseries over 24h, they are temporarily coherent. I am currently using Python /...
0
votes
1answer
61 views

Exclusion of an important predictor does not decrease random forest accuracy

I have built two binary classification models, one based on LASSO logistic regression and the second one using Random Forest. Out of 50 variables, 3 are highly correlated with each other. After ...
0
votes
0answers
24 views

What do these values from the 'importance' function mean? [closed]

I ran a random forest in R and used the 'importance' function to obtain the following values: ...
0
votes
0answers
18 views

Different results on random forest between R and Google Earth Engine

I've implemented Random Forest regression algorithm in R (randomForest package) and GEE, but they are giving me very different results (average difference is 4% o and going up to 19% in some cases), ...
1
vote
1answer
22 views

Doesn't separation in data affects Random Forest?

I'd like to know if separation in data points given by a certain variable will affect Random Forest in terms of "being a correct built model"? For example, logistic regression is highly vulnerable in ...
0
votes
0answers
15 views

Tree-based machine learning algorithms and qualitative predictors with a lot of unordered categories

As far as I understood, tree-based methods are based on yes/no questions that refer to single predictors. E.g. is predictor A larger than 3000 or is predictor B = "TRUE". In case you have a ...
0
votes
0answers
18 views

Confusion about refit model after cross-validation [duplicate]

I am kind of confused about when it is appropriate to use refit in machine learning method, say if I train a model and select the optimal hyperparameters with 5-fold cv. The highest cv score will tell ...
1
vote
1answer
63 views

When re-fitting XGBoost on most important features only, their (relative) feature importances change

I am using 60 obseravation*90features data (all continuous variables) and the response variable is also continuous. These 90 features are highly correlated and some of them might be redundant. I am ...
0
votes
0answers
14 views

Confidence Intervals for the Classification Accuracy

I am developing a classification system and after some iterations I settled on a Random Forest algorithm as the final predictor. I would like to have the confidence intervals for the estimated model's ...
2
votes
1answer
28 views

Why does a class weight fraction improve precision compared to undersampling approach where precision drops?

I have an imbalanced data where the ratio between positive to negative samples is 1:3 (positive samples are 3 times higher than negative). For my case it is is important to have a higher precision (...
0
votes
1answer
14 views

Will Random Forest be hurt if feature dependence within each group are disrupted?

I am trying to create the dataset through simulation based on known empirical features of each groups (actual training data is not available) and build a RF classifier using these simulated data. The ...
0
votes
1answer
13 views

Importance function random forest to create a logistic regression

I have created a random forest function, the dependent variable is a binary variable (either 1 or 0). When I do the importance of my random forest, it gave me the %IncMSE. Regarding it is logistic ...
0
votes
0answers
55 views

Best way to remove multicollinearity and feature selection for binary classification problem?

I am having around 1200 features 20k observations. Objective is to get the not highly correlated best 100-130 features to build binary classification models such as LR, hypertunned ML trees etc. ...
0
votes
1answer
36 views

Random Forest classifier with mean accuracy of 1? Sounds fishy

I have a small dataset with many features, but unfortunately only 19 observations from 2 categories. The idea is that I can determine feature importance in classifying samples in one of 2 categories. ...
1
vote
1answer
28 views

How can I separate the overall variable importance values when using Random forest?

I implemented a random forest model in R using the package 'ranger' combined in 'caret' package with 10fold CV. My outcome is binary (0,1) and I have a couple numeric predictor variables. I used the ...
0
votes
0answers
14 views

Random Forest with high number of categorical features?

I have a housing market data frame with the columns house_price, surface_area, bedrooms, rental_agency, postcode ,furnished, inclusive and I have a university ...
0
votes
0answers
20 views

Linear Discriminant Analysis with Random Forest

Does it make sense to combine the 2? I'm testing out several model combinations. I used LDA (Linear Discriminant Analysis) as a dimensional reduction method and layered a SVM model for classification -...
0
votes
0answers
18 views

R: Random Forest with count-data: hand it over as a quantitative integer or an ordered factor?

I am using the "rf" method as implemented in the caret package for R. In R you can differentiate between qualitative and quantitative variables. I am unsure how to view count-data. E.g. the number of ...
2
votes
2answers
90 views

One hot encoding of a binary feature when using XGBoost

I already asked this question is SO; however, I realized that this may be a better place for this type of question. I am well aware that when using categorical features with tree based models such as ...
0
votes
0answers
8 views

What statistical classifiers can use unlabelled data to enhance their performance similar to a transductive support vector machine?

I was wondering if statistical machine learning methods like tree based methods, ANNs, logistic regression can make use of unlabelled data to enhance their performance similar to the way a ...
0
votes
0answers
12 views

Imbalance class data resample gets results in overfitting Random Forest

I am working with a very imbalanced dataset (16k lines, 4% in the minority class), using random forest to for a binary classification. I’m using the Python Sklearn implementation of ...
1
vote
2answers
39 views

Is test set required to test on model after cross validation?

I understand the situation where we perform cross validation. We keep a split of the data aside never to be touched during cross validation, and test the model obtained from cross validation on the ...
0
votes
0answers
6 views

What does error rate means in a random forest model?

I have found an error rate of 31.77% for my random forest model. I have grown 10,000 trees. I don't understand what does an error rate means in a random forest. Thank you for your help :)
1
vote
1answer
34 views

Why would my random forest create a model that has a worse F1 than one of the features has on its own?

I am trying to detect a rare event (eventA) using a random forest in R. One of my features is itself an event (eventB). If eventB happens it should cause eventA, so it is a really good predictor. ...
1
vote
1answer
46 views

Am I overfitting my random forest model (more information in description)?

First off, sorry if this a novice question! Relatively new to all this stuff. Anyway, I'm working with 22 datasets that each have 180 observations of "Oddball" data and 720 observations of "Standard" ...
0
votes
1answer
33 views

Feature selection, which technique is better and why?

I have been using the techniques of Random Forest Feature Importance and Chi-square, and I would like to know if there is a reason why RF in feature selection would perform better than Chi-square. I ...
0
votes
0answers
19 views

Repeated results in simulation study of Random Forest with no seed set

Originally posted on Stack Overflow, suggested to move here. I am running a simulation study using the binary classification sets in Breimans paper. I am comparing multiple methods and looking at the ...
0
votes
0answers
7 views

Survival Analysis/Customer Attrition Model for Balances

I’m looking to model balance, meaning dollars, decay(attrition). In brief, I have time series customer data, which is aggregated to get customer’s balances for a specific product. Over time, the ...
0
votes
0answers
21 views

How does decision tree divide numerical feature? [duplicate]

As Shown in above decision Tree, sklearn's DecisionTreeClassifier divide numerical features to create decision tree. Petal length feature has following properties: ...
1
vote
0answers
12 views

Tree Based models vs traditional regression at identifying sub populations

Somewhat of a follow up to this question I asked. Can someone provide a good non technical explanation of why tree based methods are becoming so popular for identifying sub populations of people who ...
3
votes
1answer
136 views

Should the residuals of a machine learning regression model be i.i.d.?

This is a basic question but I did not find the answer in most common statistical learning books. In linear regression we assume that the residuals are i.i.d. Do we assume the same for a regression ...
1
vote
1answer
66 views

Problem with overfitting of RandomForestClassifier (Sklearn)

I have the following problem and I'm desperately looking for a solution since several weeks, so I'm hoping to find some help here. In this post, I'm always refering to Sklearn. My goal: I want to ...
2
votes
0answers
25 views

When does random forest feature importance fail?

I'm curious about the assumptions of random forest feature importance. In this paper, the author says that "We show that random forest variable importance measures are a sensible means for ...