Questions tagged [random-forest]

Random forest is a machine-learning method based on combining the outputs of many decision trees.

1
vote
1answer
11 views

why does random forest trees need to be deeper than gradient boosting trees

in Elements of Statistical Learning chapter 15. Random Forest, we see authors' note on RF v.s. GBT. One of them is that at 1000 terms, GBM depth 4 has smaller error than RF depth 6. Also we notice RF ...
0
votes
0answers
24 views

Random Forest in R. Dataset not symmetrical [on hold]

I did a couple of training with RandomForest in R for a class problem (event must be 1 or 0). Dataset consists of 10.000 rows and 10 variables (more or less) and each variable is built like ...
3
votes
0answers
14 views

OneVsRestClassifier and predict_proba

I have an interesting problem. I am working with a MULTICLASS problem (~90 classes), and have settled on using OneVsRestClassifier wrapper around a RandomForestClassifier. When I call a ....
1
vote
1answer
14 views

Model Size of Random Forest VS CatBoost

I trained models based on the same dataset, using random forest (sklearn) and CatBoost. I use n_estimators=1000 for random forest, and n_estimators(iterations)=1000 for CatBoost. The random forest ...
0
votes
2answers
53 views

Using bagging and random forests together

I was looking at a kernel implementation (for text classification) and the following piece of code got me a little bit confused (I removed part of the features - in order to keep it light - as most of ...
-1
votes
0answers
6 views

Product Prediction to group of customers

I have multiple groups of customer, say for segment 1 as shown in the pictures, I have a list of products that I can choose the cross-sell to that group. Consider ...
-1
votes
0answers
35 views

Online training with Random Forest Classifier [on hold]

Suppose I have a huge set of training data related to online transactions. I have trained a random forest classifer on full set of that transaction data. Now, consider an online learning kind of ...
2
votes
4answers
68 views

imputation FOR random forests

I was wondering what imputation method you would recommend for data to be fed into a random forest model for a classification problem. If you google for "imputation for random forests", you get a lot ...
0
votes
1answer
20 views

MissForest for SurveyData

Hello fellow data scientist, I currently reading the paper by Stekhoven & Brühlmann about MissForest. I was wondering how to deal with variables that are restricted by domain knowlege. I.e. no ...
1
vote
1answer
138 views

Fitted RF: difference between the probabilities in $votes and predict (type=“prob”)

Say we have a data frame df where diagnosis is the first column. There are only 2 possible values for ...
2
votes
1answer
42 views
1
vote
0answers
18 views

R: rpart or random forest for datasets with multiple rows per subject [closed]

I have some fundamental understanding problem with rpart or train(method="rf") in R. My data is currently structured as follows: Around 100 subjects, each has 2048 rows (so around 204,800 rows) with ...
3
votes
1answer
24 views

Predict function and number of trees

When using the predict function with a random forest regression model, do i still need to use an odd number of trees to ensure repeatable results? I understand that to be the case when there are ties ...
1
vote
0answers
25 views

Likelihood Ratio Test Equivalence for Machine Learning Methods?

I have two models that were generated using the random forest algorithm. The first uses 10 demographic predictors and the second uses the 10 demographic + 3 economic predictors. I am using the Area ...
1
vote
1answer
33 views

Same algorithm gave very different metrics on similar datasets [closed]

I am fairly new to ML and still in the learning phase. I used Random Forest ( hypertuned the parameters) for a binary classification problem on one dataset ( dataset A). I got a F1 score of 0.78. I ...
0
votes
1answer
29 views

Best way to split real valued imbalanced distribution into finite number of classes

What I'm trying to do is adapting this research paper to another problem. In short: the authors split price variations of S&P500 index into four different classes. Then they train a Random Forest ...
0
votes
0answers
12 views

Analyzing unbalanced/correlated data with classifier (behavioural biology example)

This is a real problem taken from behavioral biology. To make it simpler I will describe the specific problem instead of generalizing the description. There are multiple sound recordings of two ...
0
votes
1answer
20 views

Combining random forests and neural networks

Let's suppose the following toy example: we are given the task of estimating how many years a person has yet to leave. For this problem we have tabular data such as age, height, ethnicity, etc; and ...
0
votes
1answer
27 views

How valid is this Stacking Model (input features to weak learners are different)?

I have a set of features with 6 of them being categorical, 1 continuous and 2 textual in type. I have to predict the labels ( 10 in number) for them. I tried applying several models and came to a ...
0
votes
1answer
101 views

Machine learning when there are two “answers”

I have a problem that I am trying to use machine learning on. At a very high level, I am looking to do a transformation where in the training data I have x and it ...
0
votes
0answers
11 views

What is the formula that is used to calculated the MSE with Random Forest regression in R?

I am using the package randomForest in R for panel-data on conflict intensity. The dependent variable is the conflict intensity (e.g. the number of battle deaths). Independent variables are population,...
1
vote
1answer
30 views

How to interpret Random Forest variable importance vs. distribution of min depth plots?

I am using Random Forest (regression) to analyze data on civil conflict. I have plotted two different things: variable importance and the distribution of the min depth (using the package randomForest ...
0
votes
0answers
27 views

Will out of bag error always be 100% for classes of size 1 using random forests

My understanding is that given a sample data point $x_i$ the OOB prediction for this point will be calculated using only trees in the ensemble that do not contain $x_i$ in their bootstrap sample. Thus ...
1
vote
0answers
32 views

Features with missing entries are different in train data than in test data

I know there is a number of approaches to preprocess training data with missing entries: dropping features, imputing mean values, etc. I've compared few of such approaches and found that dropping ...
0
votes
0answers
23 views

How to compare Accuracy of two RandomForest models? (Chi Sqr or Cohen's H?)

I've got two dataset which have exactly the same structure (15 features, 1 class variable with 7 categories) and roughly the same amount of observations). I trained a Random Forest with the full ...
0
votes
0answers
15 views

Random Forest classification based on count data as features

I'm implementing a Random Forest for binary classification and I use a combination of features. Very important features are based on count statistics. I'll give an explanation: for every instance of ...
0
votes
0answers
12 views

synthesized data to train classifier

Our dataset is relatively small (303 x 14) and so we decided to use synthpop package in R. The basic idea of synthetic data is to replace some or all of the observed values by sampling from ...
0
votes
0answers
8 views

Oversampling in Uplift Modelling

I hope any of you can help me in the following matter: I am about to write my master thesis addressing the question how response and uplift modelling differ in terms of performance but also the ...
1
vote
1answer
47 views

Can Random Forests deal with variables that affect each other (and respect their temporal sequence)?

I'm studying temporal aspects of a decision-related brain process. For this study I want to see how much the information (absolute amount of contrast and/or amplitude) of a stimulus at different time ...
0
votes
0answers
31 views

Are feature importances from tree based models directly actionable for business?

If my response variable say is "has_repurchased" [0 or 1] and I have all customer level features. Can I rank the features in order of importance from the random forest model and report them as whats ...
5
votes
0answers
54 views

Asymmetric or unequal misclassification costs in random forest

I have a general question about asymmetric costs. In machine learning problems, there are times when the cost of a false positive is different from the cost of a false negative. Accordingly, models ...
3
votes
1answer
78 views

Features that are important according to random forest are not significant when logit model was used. How to interpret?

I have a feature set for each customer [age, gender, income, lifestyle, & so on...] and a response variable say: has_repurchased. I use a logit model summary which shows income & gender to ...
1
vote
1answer
56 views

How to visualize proximity score in Random Forests

For a Random Forest, we can construct a N x N (where N is the number of data points) proximity matrix ...
0
votes
0answers
31 views

Sensitivity Analysis with categorical predictive variables in R

I am doing a project where I have to predict the Sales Units in fashion and intend to run a Random Forest, Neural Networks and Support Vector Machine models. However, my predictive variables are all ...
0
votes
0answers
50 views

Feature Selection For Random Forest

Random Forest aims to combine many decision trees to make good predictions for testing data in regression and classification. It is an ensemble learning method. I have a dataset with 100 samples, ...
18
votes
3answers
2k views

Is random forest for regression a 'true' regression?

Random forests are used for regression. However, from what I understand, they assign an average target value at each leaf. Since there are only limited leaves in each tree, there are only specific ...
0
votes
0answers
16 views

Effect of the presence and absence of predictors to the classification outcome

I have trained a classification RF model and then ranked the predictors based on their contribution (importance) to the model. How can I estimate the effect of each individual predictor in the ...
1
vote
0answers
17 views

Random Forrest data prediction (in R) and data bias

I have a data set from 2014 to the present and I am trying to classify stock performance based on whether the stock outperforms the market by 5% or not (1 vs 0). I am using a random forest model in R (...
1
vote
0answers
44 views

missForest Data imputation vs. MICE using RF as imputation method?

Is the missForest package a special case of MICE using Random Forest as imputation (for just a single imputation)? The missForest algorithm is described here: https://academic.oup.com/bioinformatics/...
1
vote
2answers
87 views

What is the difference between overfitting and “not learning”

I am trying to build a Random Forests (RF) model using around 2000 observations and a number of features (can be 50 or can 1000, I still do not know which features are to be used). One way to ...
0
votes
1answer
35 views

Replacing CNNs with Random Forests

Suppose I have a sequence like "ADTGESW". Each character in this sequence can attain a number of possible values, let's say 10. I can then one-hot encode this sequence and obtain a matrix with shape ...
0
votes
0answers
21 views

I understand over-fitting in RandomForest algorithm is taken care by bootstrapping & bagging but what happens if we prune the trees and apply bagging? [duplicate]

According to Random Forest algorithm, tress are not supposed to be pruned intuitively, don't bagging on pruned models give a better final model than the one without pruning?
1
vote
0answers
33 views

Questions about confidence interval and presenting metrics for classification problem

A friend of mine and I are working on our bachelor's thesis and we're getting close to its completion. We have a few disagreements about how to present the result metrics for the experiment that we're ...
0
votes
0answers
7 views

Query Tuning RF and Neural network model output

I am working on UCI wine quality data set(total 5k records).Below is the small snap of the data.I have to predict quality(last col) which has scale of 1-10. I am trying to see the accuracy- using ...
1
vote
1answer
210 views

Combining Random forest with Adam (or an other gradient method)

There is no "gradient" in the standard Random Forest formulation, but can I combine random Forests with an optimisation method like Gradient Descent or SGD? Can I use Adam (Adaptive moment estimation)...
1
vote
1answer
64 views

What exactly is the extratrees option in ranger?

As far as I understand the splitrule = “extratrees” option in the package ranger is an implementation of Geurts et al (2006) extremely randomized trees. In their ...
2
votes
0answers
42 views

R, RandomForest, the meaning of nodesize parameter [duplicate]

I have read from several sources but I didn't fully get it (a similar question was asked in StackExchange 1 but the answer is not satisfying for me). I have still some questions waiting for the ...
0
votes
0answers
9 views

Multi-class machine learning classification on binary data - setting a class for unclassifiable

I have a large dataset of binary genomic data (i.e. mutation Y/N) A proportion of the samples have been classified into clusters based on presence of key mutations. I would like to use this ...
0
votes
1answer
77 views

random forest regression predicts “opposite”

I have a dataset with 70 features, which are continuous measures and are interrelated but not highly correlated ($|\rho| <.5$. I have several outcomes, which are each integer values ranging from 0-...
1
vote
1answer
58 views

One hot encoding vs apply the average of the label to each category

I have a fairly reasonably sized dataset (row>50k). And I'm looking for the best way to utilize some of the categorical columns. For purpose of this question, let's say that one of the categorical ...