Questions tagged [random-forest]

Random forest is a machine-learning method based on combining the outputs of many decision trees.

Filter by
Sorted by
Tagged with
0 votes
0 answers
6 views

Interpreting Shapley Values on Breast Cancer

I was analyzing Shapley Values on the Wisconsin breast cancer data set (binary classification). I applied it on Random Forest and on Ridge and Lasso Regression. However the summary plot seems to be ...
user avatar
  • 1
1 vote
1 answer
17 views

Binary Classification Problem with Predicted Probabilities distribution skewed

I have a balancedrandomforest model which was trained on unbalanced data (92/8) for a binary classification problem. The AUC is around 0.98 and the precision and recall are also acceptable being 0.89 ...
user avatar
0 votes
0 answers
7 views

Rate vs. Count Outcome with Case Weights in Random Forest

I am doing a geographic analysis to predict rate of disease in each geographic unit based on various aggregate geographic/demographic features, without any individual-level data. Each geographic unit ...
user avatar
0 votes
0 answers
25 views

When we use k-means clustering with Light GBM, comparing with Random Forest

I am developping the prediction model with many parameters. As I was not satisfied by the performance of Random Forest Regression, I tried to use k-means clustering to regroup the similar variable and ...
user avatar
0 votes
0 answers
11 views

95% confidence interval for the goodness of fit scores in regression

I see the following computation available online in classification setting. ...
user avatar
2 votes
0 answers
54 views
+50

Are there smooth analytical penalty on leaves sizes for decision trees?

In a decision tree, when we search for an optimal split, we usually minimize root mean square deviation (RMSE). In addition to that we might forbid splits that give too small leaves (for example a ...
user avatar
  • 463
2 votes
1 answer
78 views
+500

How to force splits in decision tree to be distributed uniformelly in case of no dependency on feature?

I have targets ordered by a feature. I want to find a single split that minimizes a squared deviation (RMSE). For example, I have 100 values (targets) and it might be the case that, if I split them as ...
user avatar
  • 463
1 vote
0 answers
19 views

Why am I getting 100% accuracy for SVM, Random-forest Classifier and Logistic Regression?

I'm using an existing disease prediction model to build a chatbot. While I was referring to the model I realized that it has an accuracy of 100%. I'm not quite sure how and why the accuracy is 100%. I'...
user avatar
  • 21
1 vote
0 answers
22 views

Why is the model getting 100% accuracy for SVM, Random-forest Classifier and Logistic Regression? [closed]

I'm using an existing disease prediction model to build a chatbot. While I was referring to the model I realized that it has an accuracy of 100%. I'm not quite sure how and why the accuracy is 100%. I'...
user avatar
  • 21
0 votes
0 answers
24 views

Rules, theoreticial basis on selecting which machine learning model acceptable to be combined into a voting classifier

Background: Voting ensembles (hard/maximum voting, averaging/soft voting) and stack models are considered as the ensemble technique that can improve individual performance of the machine learning ...
user avatar
1 vote
1 answer
16 views

Random Forests- Out of Bag Error Calculation

I was learning about the Out of Bag error in random forests and I did not understand a point about the error calculation. Assume we have N bootstraps and there are a number of Out-Of-Bag samples for ...
user avatar
0 votes
0 answers
9 views

learning curve for RF and LR comparison and selection

I am plotting learning curve to check how the model perform on training data set and the effect of the training size on the accuracy. I am using two models, random forest and logistic regression. From ...
user avatar
1 vote
0 answers
18 views

features from random forest classifier vs regressor

I have built random forest classifier and regressor models on same data, target and independent variables. For classifier, I am giving a classification parameter that divides the data into more or ...
user avatar
0 votes
0 answers
15 views

How to perform variable/feature selection before random forest in R?

I have a phyloseq object with 4000+ taxa and 300+ samples. What methods/packages can I use to perform feature selection to reduce the number of taxa/OTUs prior to performing random forest? I was ...
user avatar
1 vote
1 answer
13 views

Household vehicle ownership allocation based on vehicle sales data

I have a household travel survey data, which consisted of the number of vehicle owned for each of the household, household income, job, and household size. However, the data didn't show the brand, ...
user avatar
  • 11
1 vote
0 answers
36 views

Interpreting the variance of feature importance results with each random forest run using the same parameters

I noticed that I am getting different feature importance results with each random forest run even though they are using the same parameters. Now, I know that a random forest model takes observations ...
user avatar
  • 39
0 votes
0 answers
32 views

Random forest : tune and test with out-of-bag (OOB) error and data spliting

I would like to perform Random Forest (RF) with a few samples (68 observations to be exact) using r-caret package and the "ranger" implementation on continuous data. So my strategy is to ...
user avatar
0 votes
0 answers
32 views

What could cause regression linear models to predict exactly the mean of train set while random forests perform worse?

Data set: I'm working on a linear regression problem where my train set $X$ is of shape $(703 557, 53)$. Each row is a client's features, which could be its age, its gender, how many calls we received ...
user avatar
  • 155
1 vote
1 answer
23 views

Valid to compare variable importance ranks across RF with different responses?

I have a dataset with multiple response variables that share the same predictor set (in particular, semantic differential scales in an attitudes task). I want to find the predictors that best explain ...
user avatar
1 vote
1 answer
38 views

Why is a random forest regressor better than a random forest classifier when predicting a category?

I am building a model that recommends the optimal golf club based on data I have gathered. Since the model prediction should be a category, ie. a golf club, I would assume I would have to use a ...
user avatar
0 votes
0 answers
11 views

Interpreting partial dependence plots for Random Forest

I've fitted a random forest model where the target variable (wine quality) can be either 0 (bad), 1 (average) or 2 (good). I've created some partial dependence plots for some of the most important ...
user avatar
3 votes
1 answer
27 views

Features are Relevant for Regression but not necessarily for Classification - what to make of this?

I have used the R Boruta package to check for feature relevance in predicting log returns of financial time series, the targets being the log returns themselves (for regression) and the sign of log ...
user avatar
4 votes
1 answer
155 views

Can RandomForest multiplicatively combine features?

I have a relatively good understanding of how RandomForest mechanically works. However, here's what I want to understand: can RF model a multiplicative relationship? For example, if I have features A ...
user avatar
0 votes
0 answers
26 views

Should I use the predictions from my model or use the predict function on my model?

I have a small dataset (with about 5 covariates and 30 rows) and I am trying to make some predictions using R. I was advised to use Leave-One-Out Cross Validation (LOOCV) with a random forest due to ...
user avatar
  • 185
0 votes
0 answers
22 views

Random Forest: Variable Importance plot depicing negative increase (decrease) in MSE

I'm using a RF analysis to evaluate 75ish predictor variables pertaining to my response variable of percent functional area across some wetland sites. I selected the default ntree value of 500, and my ...
user avatar
  • 1
0 votes
0 answers
19 views

Feature importance inference with word2vec in a classification task

I have a binary classification task for tweets in which I am currently testing several models. Surprinsingly, the model that outbeated state of the art algorithms ,such as BERT or BERTweet, is a ...
user avatar
  • 11
2 votes
1 answer
26 views

Terminology of "Regression forest", "Random forest", "Decision tree" and "Regresion tree"

I am confused about the terminology of "regression forest", "random forest regression", "random forest", "decision tree" and "regression tree". As far ...
user avatar
  • 109
2 votes
1 answer
39 views

How to use time-series observations on multi-class classification problem?

I have a multi-class classification problem with time-series features. You can find an example series below. It shows the same series over time for different classes (actually, each line represents ...
user avatar
  • 53
0 votes
0 answers
32 views

Bootstrap to Statistically Compare Accuracy of Different Approaches

I am currently dealing with a multi-class classification problem. I have two different approaches (in terms of feature engineering) to this problem. Intuitively, the result is obvious. However, I want ...
user avatar
  • 53
0 votes
0 answers
28 views

Robust LMM or Random Forest?

I have non-normally distributed, heteroscedacic and autocorrelated data. To overcome these issues I would like to use robust linear mixed models with the robustlmm package in R or random forest ...
user avatar
  • 83
0 votes
0 answers
19 views

My random forrest regressor was overfitting so I tried to use randomsearchcv but I still got a worse result, what should I change? [duplicate]

I tried to fix my overfitting with randomized search cross-validation. These are my params: I set 100 estimators but that is irrelevant for the overfitting. I read log2 was best for regressors ...
user avatar
0 votes
1 answer
41 views

Feature selection with RandomForest and then retrain RandomForest using the selected features

I am trying to classify patients into 2 different groups using a random forest. The features correspond to the gene expression of individual patients. This means, that I have around 20.000 features (...
user avatar
  • 101
0 votes
0 answers
19 views

randomForest in R is including the class label as a feature prevents classifier from predicting on new dataset

So I have two datasets, og.data and newdata.df. I have matched their features and I want to use a feature from og.data to train a model so I can identify cases of this class in newdata.df. I am ...
user avatar
1 vote
1 answer
30 views

Good classifier for related (or higher level) features?

I'm trying to find good classifier to handle features that related to each other. For example: Features: gender, age, weight, height. Label: healthy or not. (Here I made up the example but hope you ...
user avatar
  • 111
1 vote
1 answer
26 views

I have set of features to relate to two different values. When I made a regressor for only one it worked well but if i use two it does not?

I have a set of 33x1 features (x) and they can be related to different two values in (y) and I have 1203985 observations. Using np.shape() you can see the dimensions of x and y. x= (1203985, 33) y=(...
user avatar
0 votes
0 answers
15 views

Regression Trees

Which is better over the other two? Random Forest, Bagging, or Boosting the tree-based method? My understanding is, that even though all three have their own preferred requirements to perform better. ...
user avatar
  • 1
1 vote
1 answer
27 views

RepeatedKfold - Use full data or only train data?

I am working on a binary classification using random forest with a dataset size of 977 records and 6 columns. class ratio is 77:23 (imbalanced dataset) Since, my dataset is small, I learnt that it is ...
user avatar
  • 1,780
0 votes
0 answers
39 views

Why does Random Forest perform worse than Bagging?

The 2 results I got for bagging and random forest are shown below. It seems that calculating mean MSE from bootstrapping also result in a lower mean MSE for bagging as compared to random forest. Is ...
user avatar
8 votes
1 answer
289 views

How to calibrate models if we don't have enough data?

I am working on random forest classifiation with a dataset size of 977 records and 6 features. However, my class is imbalanced and proportion is 77:23 I was reading about calibration of models (binary ...
user avatar
  • 1,780
0 votes
0 answers
30 views

Model average prediction - Usefulness and interpretation

I am working on binary classification problem using random forest with a dataset shape of 977,6. class proportion is 77:23. This post is born out these two posts here and here I recently ran the SHAP ...
user avatar
  • 1,780
1 vote
0 answers
34 views

How to get confidence estimate of random forest model predictions?

I am working on binary classification using random forest model with dataset shape of 977, 6. Class proportion is 77:23 I built the ML model using my input data and obtained the probabilities of the ...
user avatar
  • 1,780
0 votes
3 answers
33 views

Why does test MSE always decrease with increasing training size (and decreasing test size)?

Context: I am trying to find the best predictive model for a dataset with 1000 observations. The problem is I am not sure what the best training and test size should be. So what I did was that I ran ...
user avatar
0 votes
0 answers
36 views

CART coverall accuracy vs. RF & SVM

I am performing a supervised classification with RF, SVM, and CART algorithms. I have over 2000 training points in an area of 9,995 km². For CART, I have obtained a 'Validation overall accuracy = 1' ...
user avatar
0 votes
0 answers
25 views

Theoretical Random Forrest question - Predicting winners of a competition - Teaching model to focus on accuracy of only one class prediction

Apologies for the confusing title - I've built a random forest model that predicts where someone is likely to finish in a competition based on about 50 variables and 12,000 observations. People can ...
user avatar
  • 1
1 vote
1 answer
117 views

Why SHAP base/expected value is 0.5 for all my instances?

I am working on a binary classification using random forest model, neural networks in which am using SHAP to explain the model predictions. I followed the tutorial and wrote the below code to get the ...
user avatar
  • 1,780
1 vote
1 answer
18 views

Training twice on non-injective data

I have a large dataset of 30000 points, but most my Ys are the same while all Xs are different. Ys are from different samples, so I had means of Y for each sample and I used means alongside Xs to ...
user avatar
0 votes
0 answers
25 views

Does lime score matter for continuous variable discretization?

I am using a random forest classifier for binary classification with 977 records and class proportion of 77:23. I am using Lime explainer to explain the predictions made by the model. However, I see ...
user avatar
  • 1,780
0 votes
0 answers
16 views

Regression with count predictors and continuos response [duplicate]

I have the following 10000 x 779 data: columns 1-788, Predictors(independent variables X): count value, 0,1,2,3... column 789, Response(dependent variabl Y): continuos value from 0-1 I want to build a ...
user avatar
3 votes
1 answer
170 views

What is the use of expected value in machine learning models?

I see that we have a concept called expected value being used in machine learning (ML) models. For example, SHAP has a concept called Expected value. It means when ...
user avatar
  • 1,780
0 votes
2 answers
75 views

Why does my ROC curve have a sharp edge?

I was working on a random forest model in R and I got a ROC curve that looks like this. This is very odd since there is no curvature. The data does have mostly qualitative features with only 2-3 ...
user avatar
  • 1

1
2 3 4 5
46