Questions tagged [random-forest]
Random forest is a machine-learning method based on combining the outputs of many decision trees.
2,444
questions
0
votes
1
answer
25
views
Comparing the importance of interactions between models
I have two separate models that both use the same set of 10 continuous predictor variables.
Model_1 predicts one binary outcome (symptom_1: present vs. not present) and model_2 predicts another binary ...
0
votes
0
answers
7
views
How to model risk with presence-only data?
I'm working on a dataset of bird electrocutions. There are 300 instances of electrocuted birds and I have a range of environmental data that I want to input as my predictor variables. The aim is to ...
- 901
1
vote
0
answers
37
views
Regression ML model: Data Augmentation [closed]
I'm currently working on data augmentation to my regression problem, and a (possible) solution that came to my mind was to add a perturbed dataset to the original dataset, and hence double the ...
- 111
4
votes
0
answers
45
views
Why are approaches that approximate a random forest with a single decision not more popular?
I understand that random forests yield better performance than standard decision trees, but are less interpretable, because they do not generate a single tree. In this question, several users provided ...
- 41
3
votes
2
answers
1k
views
Can someone help me understand why the MAE, MSE and RMSE scores for my regression model are very low but the R2 is negative?
I am using a random forest regression model to make predictions and leave one out cross validation for my prediction task. I am having a difficult time understanding why my R2 score is negative when ...
- 33
0
votes
0
answers
73
views
Spatial structure in the random forest residuals
I have a data.frame which includes the response and several predictor variables. The variables were raster data which I converted them into a ...
- 111
0
votes
0
answers
19
views
Monotone constraints in decision tree regressor or random forest regression
after I've spent several weeks trying to fit a regression model to my flood damage data (x1=water height, x2=adaptation height, x3=(x1-x2), y=damage), it is now time for my very first question on ...
- 11
9
votes
2
answers
633
views
Relative variable importance/explained variation from a single model fit
I am seeking a measure of relative variable importance or relative explained variation that will apply to all types of linear and nonlinear regression models and that requires only fitting one model. ...
- 83.9k
1
vote
0
answers
19
views
Weighted bootstrap sampling vs. uniform bootstrap sampling with later weighting
Assume I have a fancy procedure $w: X \to \mathbb{R}$ to come up with weights for examples $x \in X$. Think of it as similar to the weights used in e.g. some boosting procedures.
Now, I want to build ...
- 321
0
votes
0
answers
9
views
Help needed for interpretation of mtry and MSE calculation for bagging and random forests
I have a question regarding the mtry values for the two models Bagging and Random Forests.
I applied the mtry measure for the California Housing Dataset and then for another dataset about white wine.
...
0
votes
0
answers
18
views
What is the splitting criterion in Regression trees (DecisionTreeRegressor sklearn) in the multi output case
I am using DecisionTreeRegressor and RandomForestRegressor from sklearn in a case where i have multiple output, but i did not find a reference article for the regression case (which is used by sklearn)...
0
votes
1
answer
20
views
Is it unwise to create a predictive model based off 20 independent variables when only 10 variables will be available for future observations?
I've created a predictive model which is based off a historical dataset and has 20 independent variables as the dataset set is comprised of completed projects, so have full information and dataset of ...
1
vote
0
answers
15
views
Need help to validate if my understanding is correct regarding random forest and feature importance
I'm currently doing binary classification task with about 40 features. The main goal of this task is finding important features. I built few tree based and binary classification models and one of my ...
- 11
0
votes
0
answers
11
views
Aggregating predictions on micro data
I am a machine-learning noob, so please bear with me.
I am trying to predict the aggregate number of businesses that will exit (i.e. shut down permanently) in the next quarter (or year). However, my ...
0
votes
0
answers
27
views
Tuning Random Forest results in max_features parameter taking a value of 1. Why?
I did a bayesian optimization tuning for parameters of random forest. With 200 iterations, it seems like 70% of the times, very low values (read 1 or 2) of max_features seems to produce better (...
- 31
2
votes
1
answer
79
views
Would repeating random forest class prediction 1000 times and choosing most frequent class improve accuracy?
I want to improve class prediction accuracy using random forest. I was wondering if I repeat the class prediction for each point 1000 times and choosing the most frequent answer make any sense (...
- 21
0
votes
1
answer
51
views
Performing a classification if having categorial labels and a distance matrix
I encountered a multi-class classification problem and I wonder which model would work the best in my scenario. I have around 50,000 vectors (each of size 200) with corresponding categorical labels ...
2
votes
1
answer
19
views
Ensemble learning with models of different quality. Develop a voting method that takes accuracy, F1, recall, calibration of each model into account
Lets assume I have 24 random forest models. Each of 24 random forest models produces a class prediction. I am currently using simple majority voting to select final prediction. Can someone please ...
- 31
0
votes
1
answer
63
views
Seeking recommendations for feature selection methods before applying a random forest model to high-dimensional data
I'm seeking recommendations for feature selection methods before applying a random forest model to high-dimensional data, specifically with over 60,000 features and only 1,000 samples. My concern is ...
- 1
2
votes
0
answers
20
views
What algorithm/approach is best for a multiclass classifier where there are a significant amount of misclassifications in the source data?
I am using document embeddings (100 dimensions) to train a classifier on text data, however, I am getting poor results which I am attributing to the fact that there are a large proportion of ...
- 31
0
votes
1
answer
79
views
How reliable is ```train_test_split```? Is there a way to optimize it?
Using train_test_split is a common practice while building a Machine Learning model. Nevertheless, partitioning your dataset to get train and test samples is an ...
3
votes
1
answer
51
views
What is the best way to meaningfully compare the feature importances between parametric (Logit) and non-parametric models (RFs/XGB)?
I have applied 3 different models (Logit, RF and XGB) to my dataset with the aim of investigating their individual predictive performance. I have found the feature importance of the logistic ...
3
votes
1
answer
68
views
Why don't partial dependence plots match model predictions?
Background
My training is in statistics, but I'm interested in machine learning. My models involve nonlinear relationships between 2-5 predictors and a single response (all variables continuous). ...
- 17.2k
0
votes
0
answers
24
views
Cost complexity pruning in random forests
When choosing the optimal alpha for cost complexity pruning in a single decision tree, we can directly look at the subset of effective alphas. However, in the context of random forests, there isn't an ...
- 101
2
votes
0
answers
36
views
Bias and Variance of a Honest Random Forest
I am trying to read the paper Estimation and Inference of Heterogeneous Treatment
Effects using Random Forests. In the section 3.1(Theoretical Background), page 13 paragraph 2, The authors have ...
- 75
1
vote
1
answer
27
views
How to combine outputs of 24 independently trained random forests outputs
Imagine I have 24 random forest classification models trained on classifying two classes Y=1 and Y=0. Each model learns on independent data, each model learns on same number of observations and same ...
- 31
2
votes
1
answer
58
views
In SkLearn's Random Forests, why is ```max_samples == None``` by default?
The more uncorrelated are the trees with each other, the better the results a Random Forest ensemble may get.
Sources: Nvidia, Tutorial by Tony Yiu
According to the official documentation, the ...
1
vote
2
answers
29
views
Within a Random Forest, is there a way to detect which trees are more correlated with each other?
One common piece of advice given when you train a Random Forest model (RF) is, increasing the number of trees as much as possible, until the accuracy doesn't improves anymore.
But, is quantity always ...
0
votes
0
answers
5
views
best practise for mtry and ntree random forest tuning using the ranger and caret packages [duplicate]
I am implementing a random forest model using the ranger package.
I know you cannot specify ntree's in the tune grid for caret. I have seen some suggestions to determine the best mtry values and then ...
- 1
0
votes
0
answers
13
views
Un-binning/Upsampling Ordinal Year-Bins into Individual Years for Random Forest Likert Analysis?
Question: Is it "quantitatively sound" to decompose/upsample year-bins (e.g., 2002-2006) into the component years when analyzing Likert Score data that was collected as a recollection of ...
- 1
2
votes
0
answers
32
views
How to choose the potential hyperparameters for GridSearchCV on RandomForestClassifier? Will default always be the best?
I'm fairly new to machine learning, and I know similar questions have been asked but I can't find an answer that satisfies my curiosity. I'm working on a Random Forest Classifier model in python, and ...
2
votes
1
answer
123
views
Why cannot a single decision tree represent an entire Random Forest?
I was intrigued by the reply from @JohnRos to the post Making a single decision tree from a random forest.
They say "<...> a random forest prediction cannot be represented by a single tree....
- 73
1
vote
1
answer
28
views
Small sample sizes and transfer learning
I have a very small dataset (n = 18) of patients with certain key data as well as imaging. I have performed a clustering analysis (via PCA and hierarchical clustering) to cluster these patients based ...
- 11
2
votes
1
answer
88
views
Design an algorithm to improve the hangman game for letter prediction [closed]
I'm working on an algorithm which is permitted to use a training set of approximately 250,000 dictionary words.
I have built and providing here with a basic, working algorithm. This algorithm will ...
- 121
0
votes
0
answers
7
views
Label encoding performing well despite data being non-ordinal
So I'm currently training a model where the dependent variable is continuous and 9/11 of the independent variables are categorical, some of these categorical variables have upwards of 10,000 classes ...
- 1
0
votes
0
answers
7
views
Regression models that conform to functional groupings of features
For example, suppose we want to predict y with features x1, x2, x3, x4. If I specify
...
- 11
0
votes
1
answer
27
views
Implementing Random Forest rolling window forecast in R [closed]
I want to forecast a dependent variable and I have some independent variables. First I shifted the dependent variable up, such that I have a supervised problem. For instance in January 2000 the ...
0
votes
0
answers
42
views
OOB in Random Forests - Detailed Calculation
In the book An introduction to statistical learning, it is mentioned that:
One can show
that on average, each bagged tree makes use of around two-thirds of the
observations. The remaining one-third ...
- 23
0
votes
0
answers
29
views
How does p-value play into classification model outcome?
I've created a classification model that predicts whether a design will succeed or fail. It returns the probability of success, rather than the actual classification label (e.g. 0.68 instead of 1). My ...
0
votes
0
answers
25
views
Using OOB Error Rate to improve Random Forest
I've been reading about the use of OOB Error Rate and its applications and use in Random Forest, for which some curiosity is how I could further use this value in the optimization of my own modelling ...
3
votes
1
answer
66
views
Сan a subset of features perform better than the base set
I have a theoretical question..
I have a model, let it be random forest
I take 100 candidate features and train the model
I select all the features that are important from the point of view of the ...
- 249
1
vote
0
answers
31
views
Using both Random Forest and RFE for Feature Selection and Dimensionality Reduction [closed]
I am using random forest classifier currently to do feature selection for a balanced dataset of about 18K rows and 7050 features. I recognize this is a lot of features.
I am thinking of using random ...
- 11
2
votes
1
answer
44
views
Entropy, Information Gain, etc. in Random Forests
I am new to ML, and just learned about the decision tree, and how entropy and information gain are used for a single tree. I am currently learning about random forest now, and some tutorials mention ...
- 23
0
votes
0
answers
9
views
Can correlating features change the AUROC of a model trained on randomly shuffled labels?
I did a binary classification with a random forest on my dataset, where I removed all correlating features and figured I'd try out a sort of "negative control" and shuffle all labels ...
- 1
3
votes
0
answers
161
views
Estimation of Propensity Score using Random Forests
Suppose that one has a binary treatment $Z$, and assume that $Z=1|X=x \sim Bern\left(e(x)\right)$.
Furthermore, suppose I want to estimate the propensity score by a random forest. Are there ...
- 111
1
vote
0
answers
13
views
Improve accuracy randomForest classification model [duplicate]
How do I improve the accuracy of the following data. It is from the following Kaggle competition which I am doing (despite it being closed for a school project).
...
- 11
1
vote
0
answers
30
views
Random Forest Variable Importance as a way to weight Nearest Neighbor Variables
I'm employing a nearest neighbor algorithm to find a real NFL game that is the most similar to my projected stats. Not all statistics have the same predictive-importance when projecting the outcome of ...
3
votes
2
answers
138
views
Is it necessary to do train-test split when we are interested in understanding the model rather than predicting?
In machine learning we are taught to always do validation of some sort, for instance by creating a hold out validation set that is used to test the performance of the model.
However, in some use cases ...
- 227
0
votes
0
answers
32
views
Multi-output Random Forest in R
I'm building a model in R to predict the total number of rushing touchdowns, passing touchdowns, defensive touchdowns, extra points, and field goals in an NFL game given several features. Basically ...
0
votes
0
answers
20
views
What is the correct calculation for AIC corrected (AICc) for a bagged random forest model using the Boston Housing data set? [duplicate]
This question of calculating AIC was answered for a specific linear model here: Calculating AIC “by hand” in R
The problem and solution for a linear model are as follows:
...
