Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Random forest is a machine-learning method based on combining the outputs of many decision trees.

0
votes
0answers
10 views

Will binary model be automatically mirrored if train set inverted?

Suppose we are training binary classifier to output 0s and 1s. It is trained and now returning float values y in range from 0 to 1. Now suppose we inverted the ...
0
votes
0answers
12 views

Why is the sqrt(n_features) the default maximum number of features for the best split in RandomForestClassifier? [duplicate]

Why does sklearn.ensemble.RandomForestClassifier references have $\sqrt{n}$ in the max_features implementation and why does randomForest in R seem to have the same $\sqrt{n}$ default? I am looking for ...
0
votes
1answer
11 views

How to get optimal model predictions for random forest in caret [on hold]

I have learnt a RF model and want to assess the performance (more specifically the accuracy, kappa and HMeasure) between the training set observation and the prediction only for the optimal model. <...
1
vote
0answers
28 views

calculate binomial deviance (binomial log-likelihood) in the test dataset

I'm predicting probabilities $P(Y=1)$ using a probability forest (ranger in R). I want to evaluate my predictions $\hat p_i$ in a test dataset by calculating average binomial deviance (log likelihood)....
2
votes
0answers
9 views

Partial Effects Plots vs. Partial Dependence Plots for Random Forests

One method for interpreting the relationship of a predictor X to the response variable Y in a fitted multivariate regression model is a Partial Effects (PE) Plot. This can be generated by holding ...
0
votes
0answers
38 views

Does hyperparameters of machine learning algorithms necessarily change performance? [closed]

I'm working on dataset (1100 rows and 40 000 features) with target variable very unbalanced (5% of positives). Train sample represents 80% of rows and test sample 20 %. I try to compare many ...
0
votes
0answers
22 views

How can I perform semi-supervised learning for Random forest algorithm in R language?

I want to train random forest by semi-supervised learning. And I have figured out a co-training framework for that, but I need to extract each tree and corresponding bootstrapped training dataset from ...
0
votes
0answers
18 views

Inducing relative class frequency/ soft (voting) probabilities for classification Random Forests in R [duplicate]

(The following question concerns binary classification) As discussed in other posts, when using Random Forests for classification one maybe not just interested in the output class (i.e. 0 or 1) but ...
2
votes
0answers
43 views

Why is my cross validation failing?

Intro So I am currently trying out multiple techniques of modelling a predictive model. I have 30 attributes (numeric) as input and 1 output (also numeric). At first I used a Linear regression which ...
0
votes
0answers
9 views

Workflow Design: Random Forest Regression with intent to see which independent variables are more important

I am very new in machine learning (and programming in general) so I apologise in advance if my question sounds too simple. I want to build a very simple random forest for regression (not ...
0
votes
0answers
10 views

sklearn's RandomForestClassifer's predict_proba returns almost same values

I am using RandomForestClassifier in sklearn to predict customer churn. The issue I am facing is that, when using the predict_proba method, the values returned are nearly the same or do not vary much ...
1
vote
0answers
24 views

regression model can't achieve low bias and low variance at the same time

I am running a RandomForestRegressor model on a dataset and it seems it can NOT achieve low bias and low variance at the same time. So I suspected that the input (independent) variables are not ...
0
votes
0answers
18 views

When computing proximity matrix with randomForest, why is running randomForest with 2000 trees different from running 2000 randomForest with 1 tree

I am running unsupervised randomForest package in R on some dataset to get the proximity matrix W. However, I noticed that the W I get from running the randomForest with 2000 trees is really different ...
0
votes
0answers
13 views

Read Categorical Value split in Random forest in R

I Have a dataset which contains various categorical variables and no numeric variable. I converted the variables to ordered factors by: ...
0
votes
1answer
21 views

Categorical Variables in Random Forests

I am aware that categorical variables should be one hot encoded before modeling with random Forests. But I am not entirely sure why. Lets say we have a predictor categorical variable with 7 levels. ...
1
vote
1answer
38 views

Feature importance in random forest

I need to know how feature importance in python adds up to 100. I have read other answers in stack overflow but could not get what I needed. Can anyone explain how feature importance in python sums up ...
0
votes
0answers
20 views

Recursive feature elimination with cross validation and random forest classification: I cannot interpret the result

I ran an expirement with my dataset and using the recursive feature elimination with cross validation and random forest classification in a binary classification problem. I came up with this result: ...
2
votes
1answer
46 views

Classification model accuracy, roc auc score, f1 score 100%

I am working on a binary classification problem. I have split the train set and when I evaluate the model on the validation set all metrics are 100% which is unrealistic considering that I haven't ...
1
vote
0answers
16 views

Is this random forest logical correct and correct implemented with R and gbm?

For professional reasons I want to learn and understand random forests. I feel unsafe if my understanding is the correct or if I am doing logical errors. I got a data set with 15 million entries and ...
0
votes
0answers
20 views

Obtaiting the Training Error from a RandomForestRegressionModel

I was planning to plot the evolution of my RandomForestRegressionModel when tunning the hyperparameters. One thing I would like to evaluate is the overfitting. For ...
0
votes
2answers
28 views

Which data model to use for nominal independent variables and continuous dependent variable?

I have a data set with two nominal features (which are my independent variables) and a continuous numeric output variable (i.e. dependent variable) between range of -10 to 10. What kind of predictive ...
0
votes
0answers
6 views

Calculate gini importance or MDI on the OOB data?

"gini importance" or "mean decrease impurity" (MDI) is one of the methods of calculating feature importance in tree models. This downside of this method is that it bias towards variables with more ...
0
votes
0answers
20 views

Random forest for tensors

Say we have have input tensor $X \in \mathbb{R}^{T \times N \times P}$ and output tensor $Y \in \mathbb{R}^{T \times N \times K}$, and we aim to build a Random Forests model $Y = f(X)+ \epsilon$. ...
3
votes
0answers
24 views

How bootstrap sampling cause bias in Random Forest?

I am confused how bootstrap sampling causes bias in the feature selection process of Random Forest. In this article, it says "Obviously, the bootstrap sampling artificially induces an association ...
0
votes
0answers
10 views

What validation is necessary for Conditional Random Forests?

I am using conditional random forests to rank the importance the variables in a model. For these, OOB ROC cannot be calculated (If I understand correctly). So, how may I test the validity of the model ...
0
votes
0answers
26 views

Matching new products with products those have actual sales data

New fashion products are arriving every season so I do not have historical sales data for them. Hence, I am using product attributes such as size,color, material, product group, price and some other ...
0
votes
0answers
33 views

recursive feature elimination and random forest: how to obtain robust variable importance rankings?

I am using recursive feature elimination based on random forest variable importance to select a minimal subset of predictors needed to model a continuous response. My first priority is to obtain a ...
3
votes
1answer
45 views

Random Forests interpretability [duplicate]

I have been using the sklearn RandomForestClassifier to solve a binary classification problem. For a particular sample prediction, I would like to be able to know ...
1
vote
0answers
15 views

Using Class Probabilities from Random Forest to do Ranking

Assume that we have set of documents which are being labelled as relevant and non-relevant. Can I used the class probabilities generated by Random Forest in python scikit-learn to rank the documents ...
0
votes
0answers
9 views

Random Forest in RankLib

I am building a LETOR model using Random Forest with RankLib. As we know that the Random Forest gets better with higher number of trees, I got a totally different observation in my case. The ...
2
votes
1answer
52 views

Random Forest: Number of Bags vs. Number of Trees

What are the differences between the number of bags and the number of trees in a Random Forest? For what I knew number of bags = number of tress in RF. However, in certain packages such as RankLib, ...
0
votes
0answers
13 views

Difference between Random Forest and Random Subspaces/Patches

When fitting a Random Forest model, a subset of the features is randomly considered at the splitting of each node. E.g., if $p$ is the number of features, then at each node in each tree, $\sqrt{p}$ ...
0
votes
0answers
17 views

Training of Isolation Forest with “normal” data only?

I was wondering if the Isolation Forest (IF) algorithm can also be used for novelty detection (no outliers in training data) and not only outlier detection (training data is contaminated with outliers)...
0
votes
0answers
8 views

I am building model for classification and in practice my data could be imbalance to any extend. How could I build single ml model to perform task? [duplicate]

for example: Case 1: Class A:10 Observations Class B: 100 Observation Case2: Class A: 100 Observations Class B: 10 Observations How could I build a single model (Random Forest) to perform this task?
1
vote
0answers
17 views

How Gini importance works [duplicate]

I have searched for how feature importance in random forest works, but I could not understand. Can anyone explain how it works?. How random forest selects feature importance by Gini?
1
vote
0answers
20 views

What is the relation between minimum instances per node and max depth?

In bagging and boosting models like random forest and xgboost we have hyper-parameters like minimum instances per node and max depth. If max depth is high the minimum instances per node will be less ...
1
vote
2answers
46 views

What fraction of the training set should I use? [closed]

I tried to fit RandomForestClassifier with n_estimators=500 to the training dataset which has 600,000 instances and it's taking ...
0
votes
0answers
17 views

measure for prediction error in random forest regression

I would like to know, if there are recommended papers describing the use of methods and measures for computing the prediction error of random forest predictions. Should I use 10-fold cross validation? ...
2
votes
1answer
44 views

classification on imbalanced dataset via random forest: results vary with random seed

I have a highly imbalanced dataset of about 8000 observations, with 11 features and one binary target variable. I want to predict the target labels, considering that the "1" target label occurs for 1....
1
vote
2answers
80 views

Handle Categorical Variables in Machine Learning in Python [closed]

I have $4$ variables in the data-set, each has more than $50$ levels in them. I want to include all these variables in my predictive model. How should I handle these categorical variables? If I do ...
0
votes
1answer
41 views

Random Forest Recursive Feature Elimination giving me different rankings

So I am trying to use RF recursive feature elimination to extract the most predictive features from my data-set. I've gotten the code to run fine and it gives me a nice table of rankings. However, ...
0
votes
0answers
11 views

contingency tables with no False Positives

I am working on a binary classification using RandomForests with repeated hold-out sampling (2/3 Train, 1/3 Validation). ...
6
votes
1answer
37 views

Why do CNNs conclude with FC layers?

From my understanding, CNNs consist of two parts. The first part (conv/pool layers) which does the feature extraction and the second part (fc layers) which does the classification from the features. ...
0
votes
1answer
32 views

Aggregation of “tree results” in random forest regression

I would like to know how the results of different decision trees are aggregated (average) in random forest regression. If I have a numeric target variable and 10 predictors, each decision tree is ...
0
votes
0answers
8 views

improve model roc_auc score

From the GridSearchcv on a random forest classifier, the best parameters is giving me an auc_roc score of 0.80. But when i train a new random forest model with the best parameters i am getting an ...
1
vote
1answer
47 views

Should one binarize qualitative variables before applying a random forest?

On which of theses two kinds of sample would a Random Forest (and more precisely sklearn RandomForest algorithm) give the best results ? (Y and other_features are continuous numerical variables, and ...
0
votes
1answer
23 views

cluster analysis in R

This is (a piece of) my data frame ...
1
vote
2answers
35 views

Variables reduction required for Random Forest, Boosting, L1, L2 regularization

I have close to 10,000 variables. I know how random forest/XGB picks number of variables randomly for building the tree. Also regularization takes care of significance of variable by its coefficient. ...
0
votes
0answers
26 views

Overfitting, cross validation and validation curve

Some time ago i used to choose my hyperparameters only relying on the Test score returned by cross validation. But after reading How does cross-validation overcome the overfitting problem? i doubted ...
4
votes
3answers
54 views

Can training data be extracted or recovered from random forest models?

I'm currently working on a random forest classifier which is trained on sensitive data. I haven't been able to find specific answers to the following questions: Can someone with access to the machine ...