Skip to main content

Questions tagged [random-forest]

Random forest is a machine-learning method based on combining the outputs of many decision trees.

Filter by
Sorted by
Tagged with
0 votes
0 answers
22 views

How Random Forest handle missing value in sk-learn? [duplicate]

What is the technic used in Random Forest Regressor from scikit-learn to handle missing value ? First I thought that a Random Forest regressor was able to natively handle missing value during training ...
Maxime Charrière's user avatar
3 votes
1 answer
39 views

Can I use simulated data only for testing a Random Forest regression already trained on real data?

I am working, using Python, on a Random Forest Regression for the prediction of a target variable. I have trained it and tested it on real data, obtaining satisfying results. Now, I would like to ...
Ismaela Avellino's user avatar
0 votes
0 answers
15 views

Feature selection for logistic regression and random forest (using Orange - no coding)

I’m using Orange to create a prediction model for the Indian liver patient dataset (binary target variable – either has or does not have liver disease – with 580 instances and 10 features). I’m using ...
Jess's user avatar
  • 21
1 vote
1 answer
18 views

Why RandomForestRegressor.score() return a coefficient of determination? [duplicate]

In ScikitLearn's method RandomForestRegressor.score(X, y), the coefficient of determination R_2 is returned as a metric of the ...
Maxime Charrière's user avatar
0 votes
0 answers
20 views

Kfold cross val in Regression model

How to use K-fold CV to evaluate my regression model performance to calculate the R2, MAE and MSE in the train set to make the model more robust? This code below refers to the tuned model and I'm ...
Vinicius Maia's user avatar
1 vote
1 answer
26 views

how to train and hypertune a model

As I am new to machine learning, and learning it myself, pardon me if I ask a silly question. My question is: What is the correct approach to building a model for, say, random forest and tuning ...
NEERAJ YADAV 's user avatar
4 votes
2 answers
66 views

If R2 is not appropriate for non-linear ML algorithms such as Random Forests, can a Pearson or Spearman correlation be used as performance metric?

$R^2$ is not appropriate for non-linear models, such as Random Forest (RFs) models. https://arxiv.org/pdf/1611.03063 Is R-squared truly an invalid metric for non-linear models? https://...
JElder's user avatar
  • 1,027
1 vote
2 answers
52 views

Is duplicating dataset an augmentation?

For a very small dataset, there is a lot of overfit in the random forest regressor model. I have removed extraneous data, scaling and feature selection, but overfit is still there. The oversampling ...
Erfan Mollai's user avatar
1 vote
0 answers
17 views

Fixed-effect trained model inspection in mixed-effects random forest (MERF)

I have run a Mixed-Effects Random Forest (MERF) using the python merf module, see therein example use here (see also blog post). I have read the above and also Hajjem et al's paper, to get an idea of ...
Emma Wiik's user avatar
1 vote
0 answers
32 views

Significant performance drop between train and validation set

I am trying both Lgbm and RandomForest for a classification, and I observe the same problem. I am using various metaparams to prevent overfitting, such as max_depth, num_trees (keeping it small for ...
Baron Yugovich's user avatar
0 votes
0 answers
15 views

Enforcing symmetries "for bag-of-vector" data in XGBoost or random forest - geodata example for illustration

I'll give a concrete toy problem, then give some comments on what sorts of abstractions I care about. Toy problem: Each person $i$ in my dataset has a phone, and every once in a while the phone will ...
user1557414's user avatar
1 vote
0 answers
25 views

What is the difference between model\$pred and model\$finalModel\$votes in a random forest model trained by caret?

I trained a random forest model as below: ...
Robin's user avatar
  • 119
4 votes
2 answers
534 views

Overfitting in randomForest model in R, WHY?

I am trying to train a Random Forest model in R for sentiment analysis. The model works with tf-idf matrix and learns from it how to classify a review, in positive or negative. Positive ones are ...
Anisa's user avatar
  • 43
0 votes
0 answers
20 views

How do I interpret a Random Forest Survival C-index value relative to the Requested performance error?

I'm doing a random forest survival analysis for a school project and I'm confused about the C-index output that I'm getting relative to the Requested performance error. Shouldn't my C-index get higher ...
Jake S's user avatar
  • 41
2 votes
0 answers
69 views

Causal forests for causal interaction effects between two treatment factors

I'm analyzing a survey experiment data with a factorial design with $2 \times 2$, where each factor is randomly assigned with equal probability. I'd also like to know the heterogeneous effect of the ...
Jin's user avatar
  • 21
2 votes
1 answer
17 views

weighted random forest with train/test datasets

I have a dataset where the sample distribution does not match the population distribution, but I have weights that can be applied to address that issue. I have randomly partitioned the original ...
Sarah Hardy's user avatar
1 vote
0 answers
45 views

Prediction vs confidence intervals using random forest / an ensemble of estimators

Given a random forest (or any other ensemble) where each of the $i=1..n$ trees/base estimators is trained by minimizing the mean squared error, then each tree/base estimator prediction $\hat{Y}_i(x) =...
Ggjj11's user avatar
  • 1,543
1 vote
1 answer
41 views

Strange increasing in $R^2$ when MAE and RMSE worsened for OLS

I am currently working on my thesis, which involves using machine learning to predict non-stationary and seasonal time series. I am encountering some results that I cannot explain. While I cannot go ...
M. Hansen's user avatar
0 votes
0 answers
11 views

Struggling with normalization/Standardisation for machine learning dataset [duplicate]

Sorry for what is probably a very obvious/rookie question. I'm currently doing a data science module for my degree and making very slow progress with the work. The case study i'm doing is around HR ...
Alex Ferry's user avatar
1 vote
1 answer
29 views

Does It Make Sense to Use Random Forest When Predictor Values Are Averaged by Groups?

In a nutshell: does it make sense to use Random Forest for analyzing the importance of predictors when predictor variables are averaged by groups of samples? I'm working on an ecological data analysis ...
Jose Antonio Morillo Perez's user avatar
0 votes
0 answers
10 views

Feature dependence in Random Forest application

I'm applying a Random Forest Regression on target variable y, number of items bought At the time of running the regression, I will have access to the 'running total'...
april-11's user avatar
0 votes
0 answers
19 views

Derive elasticities from partial dependences

I am producing a partial dependences plot based on a random forests (RF) binary classification model, which yields predicted probabilty of observation $i$ belong to class A for the outcome variable Z. ...
Giacomo's user avatar
0 votes
1 answer
49 views

Learning Curve to Know Underfitting or Overfitting

I want to know if the model I am using tends to be overfitting or underfitting. I am using SVM and Random Forest algorithms. How to figure it out?
Anna's user avatar
  • 3
0 votes
0 answers
9 views

Pooling survey years in random forests

I am currently trying to predict household wealth in a random forest, using survey data. My problem is the number of the observations. I have the option to either choose single years (n=4.000) or ...
Matzigurion's user avatar
1 vote
0 answers
32 views

Changing OOB scoring metric for RandomForestRegressor from r2 to MSE

In sklearn's documentation https://scikit-learn.org/1.4/modules/generated/sklearn.ensemble.RandomForestRegressor.html it states that the default scoring for OOB samples are r2. It also states that you ...
Mathias Nissen's user avatar
0 votes
0 answers
18 views

How can the random forest algorithm deal with the fact of not having strictly independent observations?

I have a data set with calls from two species of birds of the same genus. For each species there are 10 subjects and different numbers of calls for each (range of 1-8 calls per subject). Measurements ...
Carlos A. Flores's user avatar
0 votes
1 answer
36 views

Random forest cross-validaton by patient

I have a dataset of various features from 10 patients and 10 controls. Each patient has many data points. Random forest does an amazing job in predicting whether a data point is from a patient or a ...
SuperDuperMario's user avatar
1 vote
0 answers
57 views

Sensitivity analysis of features in a Random Forest Model

I have a built a large Random Forest Classifier and was able to output the feature importance as below: I understand that this importance is a based on mean decreased impurity. But how to interpret ...
Cathy's user avatar
  • 11
1 vote
0 answers
24 views

Question about using the RFpredInterval Package: How to calculate the standard deviation of data points in same terminal node as test data point

I am using the RFpredInterval package (specifically the rfpi function) to obtain prediction intervals for a random forest. I ...
user avatar
1 vote
0 answers
37 views

Help interpreting multi-class confusion matrix

I'm looking at the SAMHSA Mental Health Client-Level dataset. I did some t-SNE plots (dropping irrelevant cols, normalizing some, one-hot encoding some) of 500k rows out of 6.5mil. I'm trying to do ...
Jackson Walters's user avatar
0 votes
1 answer
53 views

Random Forest Regressor gives negative test score in GridSearchCV

I built a random forest regressor and used gridserachCV to tune hyperparameters. ...
Cathy's user avatar
  • 11
1 vote
0 answers
46 views

Non constant Feature Importance [closed]

I have a financial dataset which has 10 years worth of data. The aim is to build a regressor capable of predicting next year sales. So, if I want to predict sales for 2024, I could use data from 2023, ...
Nick's user avatar
  • 101
0 votes
0 answers
34 views

Important hyperparameters in survival random forest that need optimizing?

I use survival random forest for a relatively large dataset with 1200 obs and 40 features. I also wanted to compare 3 algorithms with each other by benchmarking. As tuning can be computationally ...
Faiza's user avatar
  • 1
0 votes
0 answers
29 views

Ensemble Random Forest Overfitting

I am running an ensemble random forest model (a newer method published in 2020). The model works by using a double bootstrapping step to balance imbalanced training data. Then you grow multiple ...
Greatwhite4's user avatar
0 votes
0 answers
21 views

Is the random forest classfier affected by related samples or biological replicates?

Correlation or collinearity between features can impact the results of random forest. So can having unbalanced data. However, I have not found a clear answer on whether having related samples can ...
Tal's user avatar
  • 11
0 votes
1 answer
53 views

Suspicious Partial Dependence Curves from a Random Forest Model

I have created several Partial Dependence Plots (PDP) for some of the top variables in my RF model. Overall, the plots seem to make sense; however, for the 'Bog" class, the probability for all ...
John Gallop's user avatar
0 votes
1 answer
65 views

Is it necessary to remove redundant variables from a random forest classification model?

I am running a random forest model in the following variables(attached): Is it necessary with Random Forests Classifier to remove highly correlated variables or should I leave the model as is? If I ...
John Gallop's user avatar
4 votes
1 answer
151 views

What are the best metrics to compare an OLS model and Random Forest model to predict house prices? [duplicate]

I am working on an assignment where the objective is to predict housing prices. My initial approach involves using an Ordinary Least Squares model. Following this, I plan to make a Random Forest model ...
Tim's user avatar
  • 273
0 votes
0 answers
43 views

What are the best ways to perform feature selection for a binary classification problem with extremely imbalanced dataset

I have a classification problem where the size of the dataset is about 1 million lines but the target group is only about 0.6% of the dataset. I have about 40 feature including both categorical and ...
peiman razavi's user avatar
2 votes
0 answers
45 views

use random forests for prediction

I am working on a project to determine the variables that better predict the binary outcome. I am using conditional random forest and permimp::permimp to assess the ...
Kate's user avatar
  • 185
0 votes
1 answer
28 views

Using XGB or a Random Forest Model for Interaction Minion to later include in Logistic Regression Model

I am currently developing a logistic regression model, I already have some expert based interactuion terms I will implement but I want to further figure out possible interactions I can implement in ...
Habenzu's user avatar
  • 63
1 vote
0 answers
40 views

Compare two classifier performances by prediction interval and probability coverage

Following a previously asked questions on prediction intervals for a logistic regression classifier, I'm currently experiencing a conundrum. I want to test a procedure to reverse-engineer the ...
D.K.'s user avatar
  • 11
1 vote
1 answer
76 views

Can you compare regression models using RMSE when samples have different proportions of zeros?

I am using the ranger package (which implements random forests) in R to build regression models of tree species' basal area, a continuous measure of abundance and ...
Jim Worrall's user avatar
1 vote
0 answers
30 views

Interpretation of partial dependence plot for explanatory variables with different distributions

I've fit a random forest regression model to predict a continuous landscape metric y. One of the explanatory variables in the model is continuous (Longitude), while another is categorical (Class) with ...
Jaywalker's user avatar
1 vote
0 answers
130 views

low accuracy in random forest model

...
Vons's user avatar
  • 224
1 vote
0 answers
55 views

How to interpret SHAP summary plot when some features represent are all negative or positive values?

I've been working on a random forest model to estimate the probability of soil erosion and used SHAP to understand the feature contributions. Typically, I expect to see a mix of positive and negative ...
J.Han's user avatar
  • 11
0 votes
1 answer
60 views

Bagging Ensemble Math

You are working on a binary classification problem with 3 input features and have chosen to apply a bagging algorithm (Algorithm X) on this data. You have set max_features = 2 and n_estimators = 3. ...
Tanjim Taharat Aurpa's user avatar
2 votes
1 answer
65 views

Purpose of Causal Discovery if I can include all variables in a Causal Forest Model for Causal Inference

It appears to me that I can include all variables in the dataset into a causal forest (whether or not they are confounders), as the model would conduct feature selection to identify important ...
Jia Yang's user avatar
1 vote
0 answers
42 views

Handling Mixed-Frequency time series data for Feature Selection

I'm currently working on a project where I aim to apply LASSO regularization and conduct variable importance analysis on WTI crude oil prices. My challenge is dealing with datasets that have different ...
Dome's user avatar
  • 21
8 votes
1 answer
163 views

Minimal number of features and observations for random forest regression analysis?

Linear regression is a suitable regression method even for small numbers of observations as long as there are enough observations per predictor (with factors 5 to 15 given as rules of thumb) and we ...
Bernhard's user avatar
  • 8,495

1
2 3 4 5
50