1
vote
2answers
58 views

How does random Forest work for regression?

I am an absolute beginner in field of machine learning, I started doing titanic assignment in Kaggle and found(read some where) Random Forest is the best fit. I started reading about random forest and ...
0
votes
2answers
152 views

When to avoid Random Forest?

Random forests are well known to perform fairly well on a variety of tasks and have been referred to as the leatherman of learning methods. Are there any types of problems or specific conditions in ...
1
vote
1answer
43 views

Post hoc selection of important features in random forest?

I want to guarantee a parsimonious random forest (few features used). What are methods to do this? It was suggested to me to get the feature importance after the model was created, and then create a ...
0
votes
0answers
22 views

How to deal with a feature relating to what type of expert labelled the data that becomes unavailable at point of classification?

Essentially I have a data set, that has a feature vector, and label indicating whether it is spam or non-spam. To get the labels for this data, 2 distinct types of expert were used each using ...
2
votes
4answers
81 views

Interpreting conflicting results from Random Forest & Logistic Regression?

I am using SKLearn and Statsmodel in python to build a RF and Logistic Regression, respectively. I have a feature that the RF indicates is important (feature importance of 0.202, closely behind #1 ...
1
vote
1answer
79 views

How do you validate your machine learning models?

I am wondering what approaches are commonly used for validating a classification or prediction models: Approaches that am using at the moment: Using truth-sets: - ROCs, Bootstrapping, Accuracy, ...
0
votes
1answer
41 views

Random forest ML algorithm suitable for use on cluster based HPC?

I have developed a script using pythons scipy package to analyse a rather large model that I wish to solve, the model contains over 12gb of data, including over 500 parameters. Now running small ...
1
vote
0answers
45 views

Random Forest regression model in R and data overfitting

I have trained my random forest model on a 74,000 training examples where each example consists of two proteins Amino Acids sequence (20 characters) and some numeric values representing the similarity ...
2
votes
2answers
118 views

random forest for large number of variables and predictions

I have very large number of variables compared to samples they are measured on. The following is example data in R. ...
0
votes
0answers
23 views

Response Range in Random Forest

I am using the random forest method (regression) to generate a predictor. Normally, I provide a response/answer along with each feature vector when I train my forest. However, for some of my ...
1
vote
3answers
116 views

Meaning of Bagged Random Forests?

I'm reading a paper that says that the authors used "bagged random forests". I couldn't understand this because as far as I know a random forest is a kind of bagging on its own. So a random forest is ...
1
vote
0answers
46 views

Random Forest and Factor Predictors [duplicate]

How do decision tree based ensembles like random forest deal with categorical ("factor") predictor variables? My guess would be that indicator variables are created for each factor via a ...
3
votes
2answers
245 views

Improve classification with many categorical variables

I'm working on a dataset with 200 000+ samples and approximately 50 features per sample. Among these 50 features you have 10 continuous variables and the rest of it are categorical variables ...
6
votes
1answer
195 views

Random forest on multi-level/hierarchical-structured data

I am quite new to machine learning, CART-techniques and the like, and I hope my naivete isn't too obvious. How does Random Forest handle multi-level/hierarchical data structures (for example when ...
2
votes
1answer
258 views

Relative importance of a set of predictors in a random forests classification in R

I'd like to determine the relative importance of sets of variables toward a randomForest classification model in R. The ...
2
votes
1answer
102 views

Imbalanced training dataset and Random Forest regression model

I have a large dataset (>300,000 observations) that represent the distance (RMSD) between proteins. I'm building a regression model (Random Forest) that is supposed to predict the distance between any ...
1
vote
1answer
95 views

building a classification model for strictly binary data

i have a data set that is strictly binary. each variable's set of values is in the domain: true, false. the "special" property of this data set is that an overwhelming majority of the values are ...
0
votes
0answers
57 views

Random forest: confounding factors

I have N variables in K samples. There is a classification variable, T (treatment), and a confounding variable -- sex. Unfortunately, in the "no treatment" (CTRL) group there are significantly more ...
3
votes
1answer
48 views

Why the trees generated via bagging are identically distributed?

I have problem in intuitive understanding of following arguement: "The trees generated via bagging are identically distributed, thus the expectation of the average of a set of trees is the same as ...
2
votes
1answer
137 views

Monte Carlo simulation vs. machine learning algorithms: what is the difference in application?

I have been doing some research on different type of machine learning (ML) algorithms such as random forest/SVM etc. in order to model and best predict pharmaceutical needs of patients suffering from ...
2
votes
0answers
41 views

Identifying what weights to give to each class in a Random Forest

I am using a randomForest package in R to discriminate between 4 categories. My data consists of 80+ observations and is heavily unbalanced with around 70% of all observations being in a single ...
0
votes
0answers
29 views

Early split decision criteria for fast random (regression) forest estimation

Suppose I am on a node in a $regression$ tree and I am using running estimates of $\sum_{i \in Region_1} (y_i - mean(y_i)_{Region1})^2$ (and the same for Region 2) to determine whether to split the ...
0
votes
0answers
66 views

what to do with 0.5 class probabilities ?

I am currently training a random forest regressor (scikit learn) on the Titanic dataset. My question is related to this issue ...
2
votes
1answer
92 views

How to incorporate constraints in random forest output

Suppose I am doing random forest classification of labels $A$,$B$,$C$,$D$. There is some theoretical ordering to this output such that when $A$ is more likely than $B$, $B$ is also more likely than ...
0
votes
1answer
76 views

Can we remove trees from a random forest with poor OOB error to improve generalisation?

My objective is to improve out of sample generalization of my random forest while holding the number of trees constant. Suppose that I am only allowed to use $n$ trees on the out of sample data but ...
1
vote
2answers
156 views

Weighting more recent data in Random Forest model

I'm training a classification model with Random Forest to discriminate between 6 categories. My transactional data has approximately 60k+ observations and 35 variables. Here's an example of how it ...
0
votes
0answers
103 views

Understanding the RandomForest with 10x10 cross validation for classification

I'm trying to understand the built of a random forest with the $10\times10$ cross validation for a binary classification problem. Therefore I have 4 basic questions: Notation: $N=500$ trees $i=$ ...
4
votes
1answer
741 views

How to choose train/test sample ratio, for machine learning?

I am building a real time machine learning module, which is not based on a huge** sample size, with hyper parameter grid search and cross validation process. I am looking for any insight/advice, as ...
2
votes
1answer
99 views

Statistical Significance of a learning Model

I built a learning model (for classification) based on a Random Forest classifier and i am asked to assess the statistical significance of its performances. Up to now, i trained and tested it on two ...
0
votes
0answers
76 views

Effect of mtry and trainingset size in Random Forest

I've plotted the learning curves below, using different RF trainers. The training set is very small and with few features (it's the popular Titanic dataset). At pre-processing stage, I just created a ...
0
votes
1answer
352 views

important features in RandomForest - Sklearn

1) How to find important features in RandomForest classifier (in sklearn) with high statistical significant? 2) The input data I have is unbalanced which I simply repeat data to compensate that. When ...
4
votes
1answer
379 views

Can Random Forests do much better than the 2.8% test error on MNIST?

I haven't found any literature on the application of Random Forests to MNIST, CIFAR, STL-10, etc. so I thought I'd try them with the permutation-invariant MNIST myself. In R, I tried: ...
1
vote
1answer
143 views

Learning curve shows decreasing accuracy

I'm working on a random forest classifier with 10-folds CV to aestimate the hyperparameter 'mtry' (chosen by maximizing AUROC). I decided to pre-split the training set in 8 samples equals in size ...
1
vote
1answer
82 views

Plot cost function for Random Forest against sample size in R

I would like to aestimate the cost function of a random forest model fed by several subsets of my training/test data. The subsets are increasing in size. Comparing the cost against the training and ...
1
vote
1answer
576 views

Does party package in R provide out-of-bag estimates of error for Random Forest models?

I'm a new R user, and also new to Random Forest modeling. I cannot seem to figure out how to obtain the out-of-bag (OOB) error estimates for cforest models built with the Party Package in R. In the ...
7
votes
4answers
811 views

Random Forest, is it a boosting algorithm?

Short definition of boosting: Can a set of weak learners create a single strong learner? A weak learner is defined to be a classifier which is only slightly correlated with the true ...
1
vote
0answers
189 views

Significance of R Squared in Random Forest / GBM and GBM Tuning Parameters

I often get different level of responses when I discuss about R-Squared and its relevance to measuring the performance of a Random Forest or GBM model. In general, RMSE is a better and more ...
3
votes
4answers
334 views

How to choose the split in Random forest for categorical predictors (features)?

I understand how best split is chosen for random forest for numerical predictors (features). Numerical predictors are sorted then for every value Gini impurity or entropy is calculated and a ...
1
vote
0answers
91 views

Organizing data to feed random forests

I'm willing to apply machine learning with R (I will start with random forests then maybe have a look at NNs) on some data, but I don't know where to start, ...
0
votes
0answers
22 views

Breiman's RF code [duplicate]

Is anyone familiar with usage of Random forest code Breiman and Cutler (http://stat-www.berkeley.edu/users/breiman/RandomForests/cc_home.htm).?? I wanted to know how to use that FORTRAN code to ...
2
votes
3answers
728 views

How to perform unsupervised Random Forest classification using Breiman's code?

I am working with Breiman's random forest code (http://stat-www.berkeley.edu/users/breiman/RandomForests/cc_manual.htm#c2) for classification of satellite data (supervised learning). I am using a ...
2
votes
1answer
380 views

Time-series machine learning methods and R packages

I am trying to determine how to use machine learning models such as for eg., random Forest with (non-financial) time-series data. Using an example, suppose we wanted to find based on monthly scores ...
4
votes
1answer
416 views

What is the objective Scikit-learn's Random Forest classifier is optimizing at each node?

I would like to ask what is the specific objective function that Scikit-learn's Random Forest classifier is optimizing at each node for the "Entropy" option. My understanding is that entropy is used ...
1
vote
1answer
3k views

Using random forest in MATLAB

I am having issues in using random forests in MATLAB. I have features of size 2000 and around 4000 data points. I am trying to learn how to compute random forests in MATLAB using the library Random ...
0
votes
2answers
77 views

Prediction with one very strong and many weaker variables

I want to create an RF model, with about 100 weak variables and one very strong variable. The strong variable is a probability score, I do not have visibility on how it was derived. It may be using ...
1
vote
1answer
113 views

What is the min_density parameter in scikit-learn Random Forest/ExtraTrees for?

The ExtraTreesClassifier and Random Forest in Scikit learn library has a parameter "min_density". Its default value is set to 0.1. I cannot seem to figure out what this min_density parameter means ...
2
votes
0answers
65 views

Out-of-bag estimate biased by correlated features

I have a data set with a small number of samples (322) and a large number of features (318.976). My data consists of images, and I want to train a binary classifier. Since I have such a small amount ...
6
votes
1answer
244 views

Motivation behind random forest algorithm steps

The method that I'm familiar with for constructing a random forest is as follows: (from http://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm) To build a tree in the forest we: Bootstrap ...
1
vote
1answer
93 views

Variable cost - benefit analysis on random forest

I have a random forest being trained with n vectors each with m variables. Each variable has a cost based on how much time it takes to compute it (m1 might take 1 unit while m2 might take 100, making ...
4
votes
0answers
142 views

Under which conditions do gradient boosting machines outperform random forests?

Can Friedman's gradient boosting machine achieve better performance than random forests? If so, in which conditions or what kind of data set can make gbm better?