Questions tagged [random-forest]

Random forest is a machine-learning method based on combining the outputs of many decision trees.

2,475 questions
Filter by
Sorted by
Tagged with
22 views

How Random Forest handle missing value in sk-learn? [duplicate]

What is the technic used in Random Forest Regressor from scikit-learn to handle missing value ? First I thought that a Random Forest regressor was able to natively handle missing value during training ...
39 views

Can I use simulated data only for testing a Random Forest regression already trained on real data?

I am working, using Python, on a Random Forest Regression for the prediction of a target variable. I have trained it and tested it on real data, obtaining satisfying results. Now, I would like to ...
15 views

Feature selection for logistic regression and random forest (using Orange - no coding)

I’m using Orange to create a prediction model for the Indian liver patient dataset (binary target variable – either has or does not have liver disease – with 580 instances and 10 features). I’m using ...
• 21
1 vote
18 views

Why RandomForestRegressor.score() return a coefficient of determination? [duplicate]

In ScikitLearn's method RandomForestRegressor.score(X, y), the coefficient of determination R_2 is returned as a metric of the ...
20 views

Kfold cross val in Regression model

How to use K-fold CV to evaluate my regression model performance to calculate the R2, MAE and MSE in the train set to make the model more robust? This code below refers to the tuned model and I'm ...
1 vote
26 views

how to train and hypertune a model

As I am new to machine learning, and learning it myself, pardon me if I ask a silly question. My question is: What is the correct approach to building a model for, say, random forest and tuning ...
66 views

If R2 is not appropriate for non-linear ML algorithms such as Random Forests, can a Pearson or Spearman correlation be used as performance metric?

$R^2$ is not appropriate for non-linear models, such as Random Forest (RFs) models. https://arxiv.org/pdf/1611.03063 Is R-squared truly an invalid metric for non-linear models? https://...
• 1,027
1 vote
52 views

Is duplicating dataset an augmentation?

For a very small dataset, there is a lot of overfit in the random forest regressor model. I have removed extraneous data, scaling and feature selection, but overfit is still there. The oversampling ...
1 vote
17 views

Fixed-effect trained model inspection in mixed-effects random forest (MERF)

I have run a Mixed-Effects Random Forest (MERF) using the python merf module, see therein example use here (see also blog post). I have read the above and also Hajjem et al's paper, to get an idea of ...
1 vote
32 views

Significant performance drop between train and validation set

I am trying both Lgbm and RandomForest for a classification, and I observe the same problem. I am using various metaparams to prevent overfitting, such as max_depth, num_trees (keeping it small for ...
15 views

Enforcing symmetries "for bag-of-vector" data in XGBoost or random forest - geodata example for illustration

I'll give a concrete toy problem, then give some comments on what sorts of abstractions I care about. Toy problem: Each person $i$ in my dataset has a phone, and every once in a while the phone will ...
• 166
1 vote
25 views

• 1,543
1 vote
41 views

Strange increasing in $R^2$ when MAE and RMSE worsened for OLS

I am currently working on my thesis, which involves using machine learning to predict non-stationary and seasonal time series. I am encountering some results that I cannot explain. While I cannot go ...
11 views

Struggling with normalization/Standardisation for machine learning dataset [duplicate]

Sorry for what is probably a very obvious/rookie question. I'm currently doing a data science module for my degree and making very slow progress with the work. The case study i'm doing is around HR ...
1 vote
29 views

Does It Make Sense to Use Random Forest When Predictor Values Are Averaged by Groups?

In a nutshell: does it make sense to use Random Forest for analyzing the importance of predictors when predictor variables are averaged by groups of samples? I'm working on an ecological data analysis ...
10 views

Feature dependence in Random Forest application

I'm applying a Random Forest Regression on target variable y, number of items bought At the time of running the regression, I will have access to the 'running total'...
19 views

Derive elasticities from partial dependences

I am producing a partial dependences plot based on a random forests (RF) binary classification model, which yields predicted probabilty of observation $i$ belong to class A for the outcome variable Z. ...
49 views

Learning Curve to Know Underfitting or Overfitting

I want to know if the model I am using tends to be overfitting or underfitting. I am using SVM and Random Forest algorithms. How to figure it out?
• 3
9 views

Pooling survey years in random forests

I am currently trying to predict household wealth in a random forest, using survey data. My problem is the number of the observations. I have the option to either choose single years (n=4.000) or ...
1 vote
32 views

Changing OOB scoring metric for RandomForestRegressor from r2 to MSE

In sklearn's documentation https://scikit-learn.org/1.4/modules/generated/sklearn.ensemble.RandomForestRegressor.html it states that the default scoring for OOB samples are r2. It also states that you ...
18 views

How can the random forest algorithm deal with the fact of not having strictly independent observations?

I have a data set with calls from two species of birds of the same genus. For each species there are 10 subjects and different numbers of calls for each (range of 1-8 calls per subject). Measurements ...
36 views

Random forest cross-validaton by patient

I have a dataset of various features from 10 patients and 10 controls. Each patient has many data points. Random forest does an amazing job in predicting whether a data point is from a patient or a ...
1 vote
57 views

Sensitivity analysis of features in a Random Forest Model

I have a built a large Random Forest Classifier and was able to output the feature importance as below: I understand that this importance is a based on mean decreased impurity. But how to interpret ...
• 11
1 vote
24 views

Question about using the RFpredInterval Package: How to calculate the standard deviation of data points in same terminal node as test data point

I am using the RFpredInterval package (specifically the rfpi function) to obtain prediction intervals for a random forest. I ...
1 vote
37 views

Help interpreting multi-class confusion matrix

I'm looking at the SAMHSA Mental Health Client-Level dataset. I did some t-SNE plots (dropping irrelevant cols, normalizing some, one-hot encoding some) of 500k rows out of 6.5mil. I'm trying to do ...
53 views

Random Forest Regressor gives negative test score in GridSearchCV

I built a random forest regressor and used gridserachCV to tune hyperparameters. ...
• 11
1 vote
46 views

Non constant Feature Importance [closed]

I have a financial dataset which has 10 years worth of data. The aim is to build a regressor capable of predicting next year sales. So, if I want to predict sales for 2024, I could use data from 2023, ...
• 101
34 views

Important hyperparameters in survival random forest that need optimizing?

I use survival random forest for a relatively large dataset with 1200 obs and 40 features. I also wanted to compare 3 algorithms with each other by benchmarking. As tuning can be computationally ...
29 views

Ensemble Random Forest Overfitting

I am running an ensemble random forest model (a newer method published in 2020). The model works by using a double bootstrapping step to balance imbalanced training data. Then you grow multiple ...
21 views

Is the random forest classfier affected by related samples or biological replicates?

Correlation or collinearity between features can impact the results of random forest. So can having unbalanced data. However, I have not found a clear answer on whether having related samples can ...
• 11
53 views

Suspicious Partial Dependence Curves from a Random Forest Model

I have created several Partial Dependence Plots (PDP) for some of the top variables in my RF model. Overall, the plots seem to make sense; however, for the 'Bog" class, the probability for all ...
65 views

Is it necessary to remove redundant variables from a random forest classification model?

I am running a random forest model in the following variables(attached): Is it necessary with Random Forests Classifier to remove highly correlated variables or should I leave the model as is? If I ...
151 views

What are the best metrics to compare an OLS model and Random Forest model to predict house prices? [duplicate]

I am working on an assignment where the objective is to predict housing prices. My initial approach involves using an Ordinary Least Squares model. Following this, I plan to make a Random Forest model ...
• 273
43 views

What are the best ways to perform feature selection for a binary classification problem with extremely imbalanced dataset

I have a classification problem where the size of the dataset is about 1 million lines but the target group is only about 0.6% of the dataset. I have about 40 feature including both categorical and ...
45 views

use random forests for prediction

I am working on a project to determine the variables that better predict the binary outcome. I am using conditional random forest and permimp::permimp to assess the ...
• 185
28 views

Using XGB or a Random Forest Model for Interaction Minion to later include in Logistic Regression Model

I am currently developing a logistic regression model, I already have some expert based interactuion terms I will implement but I want to further figure out possible interactions I can implement in ...
• 63
1 vote
40 views

Compare two classifier performances by prediction interval and probability coverage

Following a previously asked questions on prediction intervals for a logistic regression classifier, I'm currently experiencing a conundrum. I want to test a procedure to reverse-engineer the ...
• 11
1 vote
76 views

Can you compare regression models using RMSE when samples have different proportions of zeros?

I am using the ranger package (which implements random forests) in R to build regression models of tree species' basal area, a continuous measure of abundance and ...
• 111
1 vote
30 views

Interpretation of partial dependence plot for explanatory variables with different distributions

I've fit a random forest regression model to predict a continuous landscape metric y. One of the explanatory variables in the model is continuous (Longitude), while another is categorical (Class) with ...
1 vote
130 views

...
• 224
1 vote
55 views

How to interpret SHAP summary plot when some features represent are all negative or positive values?

I've been working on a random forest model to estimate the probability of soil erosion and used SHAP to understand the feature contributions. Typically, I expect to see a mix of positive and negative ...
• 11
60 views

Bagging Ensemble Math

You are working on a binary classification problem with 3 input features and have chosen to apply a bagging algorithm (Algorithm X) on this data. You have set max_features = 2 and n_estimators = 3. ...
65 views

Purpose of Causal Discovery if I can include all variables in a Causal Forest Model for Causal Inference

It appears to me that I can include all variables in the dataset into a causal forest (whether or not they are confounders), as the model would conduct feature selection to identify important ...
• 21
1 vote
42 views

Handling Mixed-Frequency time series data for Feature Selection

I'm currently working on a project where I aim to apply LASSO regularization and conduct variable importance analysis on WTI crude oil prices. My challenge is dealing with datasets that have different ...
• 21