5
votes
0answers
100 views

Random forest on multi-level/hierarchical-structured data

I am quite new to machine learning, CART-techniques and the like, and I hope my naivete isn't too obvious. How does Random Forest handle multi-level/hierarchical data structures (for example when ...
2
votes
1answer
76 views

Relative importance of a set of predictors in a random forests classification in R

I'd like to determine the relative importance of sets of variables toward a randomForest classification model in R. The ...
2
votes
1answer
48 views

Imbalanced training dataset and Random Forest regression model

I have a large dataset (>300,000 observations) that represent the distance (RMSD) between proteins. I'm building a regression model (Random Forest) that is supposed to predict the distance between any ...
1
vote
1answer
60 views

building a classification model for strictly binary data

i have a data set that is strictly binary. each variable's set of values is in the domain: true, false. the "special" property of this data set is that an overwhelming majority of the values are ...
0
votes
0answers
26 views

Random forest: confounding factors

I have N variables in K samples. There is a classification variable, T (treatment), and a confounding variable -- sex. Unfortunately, in the "no treatment" (CTRL) group there are significantly more ...
3
votes
1answer
42 views

Why the trees generated via bagging are identically distributed?

I have problem in intuitive understanding of following arguement: "The trees generated via bagging are identically distributed, thus the expectation of the average of a set of trees is the same as ...
2
votes
1answer
102 views

Monte Carlo simulation vs. machine learning algorithms: what is the difference in application?

I have been doing some research on different type of machine learning (ML) algorithms such as random forest/SVM etc. in order to model and best predict pharmaceutical needs of patients suffering from ...
2
votes
0answers
30 views

Identifying what weights to give to each class in a Random Forest

I am using a randomForest package in R to discriminate between 4 categories. My data consists of 80+ observations and is heavily unbalanced with around 70% of all observations being in a single ...
0
votes
0answers
17 views

Early split decision criteria for fast random (regression) forest estimation

Suppose I am on a node in a $regression$ tree and I am using running estimates of $\sum_{i \in Region_1} (y_i - mean(y_i)_{Region1})^2$ (and the same for Region 2) to determine whether to split the ...
0
votes
0answers
51 views

what to do with 0.5 class probabilities ?

I am currently training a random forest regressor (scikit learn) on the Titanic dataset. My question is related to this issue ...
1
vote
1answer
65 views

How to incorporate constraints in random forest output

Suppose I am doing random forest classification of labels $A$,$B$,$C$,$D$. There is some theoretical ordering to this output such that when $A$ is more likely than $B$, $B$ is also more likely than ...
0
votes
1answer
48 views

Can we remove trees from a random forest with poor OOB error to improve generalisation?

My objective is to improve out of sample generalization of my random forest while holding the number of trees constant. Suppose that I am only allowed to use $n$ trees on the out of sample data but ...
1
vote
2answers
69 views

Weighting more recent data in Random Forest model

I'm training a classification model with Random Forest to discriminate between 6 categories. My transactional data has approximately 60k+ observations and 35 variables. Here's an example of how it ...
0
votes
0answers
73 views

Understanding the RandomForest with 10x10 cross validation for classification

I'm trying to understand the built of a random forest with the $10\times10$ cross validation for a binary classification problem. Therefore I have 4 basic questions: Notation: $N=500$ trees $i=$ ...
3
votes
1answer
180 views

How to choose train/test sample ratio, for machine learning?

I am building a real time machine learning module, which is not based on a huge** sample size, with hyper parameter grid search and cross validation process. I am looking for any insight/advice, as ...
2
votes
1answer
95 views

Statistical Significance of a learning Model

I built a learning model (for classification) based on a Random Forest classifier and i am asked to assess the statistical significance of its performances. Up to now, i trained and tested it on two ...
0
votes
0answers
54 views

Effect of mtry and trainingset size in Random Forest

I've plotted the learning curves below, using different RF trainers. The training set is very small and with few features (it's the popular Titanic dataset). At pre-processing stage, I just created a ...
0
votes
1answer
133 views

important features in RandomForest - Sklearn

1) How to find important features in RandomForest classifier (in sklearn) with high statistical significant? 2) The input data I have is unbalanced which I simply repeat data to compensate that. When ...
3
votes
1answer
195 views

Can Random Forests do much better than the 2.8% test error on MNIST?

I haven't found any literature on the application of Random Forests to MNIST, CIFAR, STL-10, etc. so I thought I'd try them with the permutation-invariant MNIST myself. In R, I tried: ...
1
vote
1answer
87 views

Learning curve shows decreasing accuracy

I'm working on a random forest classifier with 10-folds CV to aestimate the hyperparameter 'mtry' (chosen by maximizing AUROC). I decided to pre-split the training set in 8 samples equals in size ...
1
vote
1answer
60 views

Plot cost function for Random Forest against sample size in R

I would like to aestimate the cost function of a random forest model fed by several subsets of my training/test data. The subsets are increasing in size. Comparing the cost against the training and ...
1
vote
1answer
222 views

Does party package in R provide out-of-bag estimates of error for Random Forest models?

I'm a new R user, and also new to Random Forest modeling. I cannot seem to figure out how to obtain the out-of-bag (OOB) error estimates for cforest models built with the Party Package in R. In the ...
6
votes
4answers
307 views

Random Forest, is it a boosting algorithm?

Short definition of boosting: Can a set of weak learners create a single strong learner? A weak learner is defined to be a classifier which is only slightly correlated with the true ...
1
vote
0answers
109 views

Significance of R Squared in Random Forest / GBM and GBM Tuning Parameters

I often get different level of responses when I discuss about R-Squared and its relevance to measuring the performance of a Random Forest or GBM model. In general, RMSE is a better and more ...
3
votes
3answers
152 views

How to choose the split in Random forest for categorical predictors (features)?

I understand how best split is chosen for random forest for numerical predictors (features). Numerical predictors are sorted then for every value Gini impurity or entropy is calculated and a ...
1
vote
0answers
85 views

Organizing data to feed random forests

I'm willing to apply machine learning with R (I will start with random forests then maybe have a look at NNs) on some data, but I don't know where to start, ...
0
votes
0answers
22 views

Breiman's RF code [duplicate]

Is anyone familiar with usage of Random forest code Breiman and Cutler (http://stat-www.berkeley.edu/users/breiman/RandomForests/cc_home.htm).?? I wanted to know how to use that FORTRAN code to ...
2
votes
2answers
274 views

How to perform unsupervised Random Forest classification using Breiman's code?

I am working with Breiman's random forest code (http://stat-www.berkeley.edu/users/breiman/RandomForests/cc_manual.htm#c2) for classification of satellite data (supervised learning). I am using a ...
2
votes
1answer
235 views

Time-series machine learning methods and R packages

I am trying to determine how to use machine learning models such as for eg., random Forest with (non-financial) time-series data. Using an example, suppose we wanted to find based on monthly scores ...
3
votes
1answer
285 views

What is the objective Scikit-learn's Random Forest classifier is optimizing at each node?

I would like to ask what is the specific objective function that Scikit-learn's Random Forest classifier is optimizing at each node for the "Entropy" option. My understanding is that entropy is used ...
1
vote
1answer
1k views

Using random forest in MATLAB

I am having issues in using random forests in MATLAB. I have features of size 2000 and around 4000 data points. I am trying to learn how to compute random forests in MATLAB using the library Random ...
0
votes
2answers
75 views

Prediction with one very strong and many weaker variables

I want to create an RF model, with about 100 weak variables and one very strong variable. The strong variable is a probability score, I do not have visibility on how it was derived. It may be using ...
1
vote
1answer
86 views

What is the min_density parameter in scikit-learn Random Forest/ExtraTrees for?

The ExtraTreesClassifier and Random Forest in Scikit learn library has a parameter "min_density". Its default value is set to 0.1. I cannot seem to figure out what this min_density parameter means ...
2
votes
0answers
57 views

Out-of-bag estimate biased by correlated features

I have a data set with a small number of samples (322) and a large number of features (318.976). My data consists of images, and I want to train a binary classifier. Since I have such a small amount ...
6
votes
1answer
162 views

Motivation behind random forest algorithm steps

The method that I'm familiar with for constructing a random forest is as follows: (from http://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm) To build a tree in the forest we: Bootstrap ...
1
vote
1answer
84 views

Variable cost - benefit analysis on random forest

I have a random forest being trained with n vectors each with m variables. Each variable has a cost based on how much time it takes to compute it (m1 might take 1 unit while m2 might take 100, making ...
3
votes
0answers
123 views

Under which conditions do gradient boosting machines outperform random forests?

Can Friedman's gradient boosting machine achieve better performance than random forests? If so, in which conditions or what kind of data set can make gbm better?
1
vote
1answer
96 views

Calibrating random forest regression

I'm trying to learn how to calibrate random forests for regression. I have found many informative sources on how to do this for classification but none for regression. The CORElearn R package has a ...
2
votes
2answers
182 views

Combining features to improve F1 score in random forest

In a project we have to use random forests to classify data into 5 sets. The data set is large (300k data elements and 9 features) and the result is based of F1 score rather than mean square error. A ...
2
votes
2answers
118 views

Random Forest and cluster-level bootstrapping

I'm working with cluster-correlated data (individuals nested into households). I would like to modify the bootstrap sampling process to make a household-level bootstrap instead of the subject-level ...
0
votes
0answers
34 views

Modelling non-independent y through a naive model by using shifted values of y as input

Excuse the unclear title, but I'm trying to ask about the validity of using a shifted dependent variable as a predictor of itself as a crude way of increasing the accuracy of predicting outcomes which ...
0
votes
0answers
102 views

Accounting for sampling bias in Random Forest Model

I know I have significant annual variation due to observer bias in my data and also unequal sample sizes between years. I wanted to account for this by using "year" as a nuisance variable in a random ...
3
votes
1answer
638 views

First steps learning to predict financial timeseries using machine learning

I am trying to get a grasp on how to use machine learning to predict financial timeseries 1 or more steps into the future. I have a financial timeseries with some descriptive data and I would like to ...
0
votes
0answers
67 views

Appropriate method for supervised learning of small data set with few variables

What method exc. for regression can be used in order to get y=f(x1,x2) on a training set of 800 to 2000 samples? y is a whole number <0,15>, x1,x2 are real <0,40>? I'm interested in prediction ...
2
votes
1answer
156 views

When to Log/Exp your Variables when performing Linear Regression?

I'm doing regression using Random Forests for predicting prices based on several attributes. Code is written in Python using Scikit-learn. How do you decide whether you should transform your ...
1
vote
0answers
74 views

Which Regression methods are suitable for binary valued features and continuous output?

I want to build a machine learning model to regression on continuous output given binary valued features(0,1). the dimension of my problem is around 200. which of the flowing methods seems suitable ...
1
vote
4answers
181 views

Measuring representativeness of a sample using covariates

I was provided with quite a small sample of labeled (variable of interest) observations to train a model to predict unlabeled observations. All the observations are associated with many covariates. ...
2
votes
0answers
108 views

Maximum number of classes for RandomForest multiclass estimation

I have researched the internet|literature a lot on multiclass prediction to find out what is a realistic limit for the number of classes that can successfully be used for estimation when using a ...
3
votes
1answer
183 views

Variable importance randomForest negative values

I am asking myself if it is a good idea to remove those variables with a negative variable importance value ("%IncMSE") in a regression context. And if it gives me a better prediction? What do you ...
1
vote
1answer
78 views

Non-independence of IVs in a random forest model

How is a random forest model affected if some of the variables are not independent?