Random forest is a machine-learning method based on combining the outputs of many decision trees.

learn more… | top users | synonyms

0
votes
2answers
21 views

Is feature engineering relevant at all for Random Forests?

Random forests is an ensemble of trees that learns the hidden patterns in the data. I have mostly tried doing some feature-engineering before running the Random Forest model but is it required or the ...
0
votes
0answers
6 views

RandomForest undersampling non events

I have data that is about 50 to 1 nonevent to event (in this case, purchases). I was taking an equal, or close to equal, sample of nonevents to events with a random forest model, and running the ...
0
votes
0answers
16 views

Need new strategy for single class classifier

I am attempting to create a single class classifier where the classes are fairly close to balanced (+/- 25). My dataset has ~2,800 samples and ~1,100 features. All of the features are binary except ...
1
vote
1answer
22 views

In a random forest algorithm, how can one intrepret the importance of each feature?

I am in the process of building a Random Forest algorithm in MATLAB using the TreeBagger function. In the documentation, it returns 3 parameters about the importance of the input features. My ...
0
votes
0answers
15 views

Random Forest: Strange Feature Importance Results for Constant Variables

I've been using the RandomForestClassifier in python's Sklearn package to assess the importance of the features in a large dataset with features that are both binary and continuous. I've done quite a ...
0
votes
3answers
35 views

Multiclass classification question

I am working on applying Random Forests to a multiclass classification problem, where I have a set of 11 predictor variables and a response that can take the values of "Yes", "No", and "Maybe". In my ...
1
vote
1answer
33 views

Time Series forecasting with useful predictor variables

I am playing with time series data related to a issue ticketing system. The system logs all open tickets at any one point and my task is to predict what the volume of open tickets will be in 5,10,15 ...
1
vote
0answers
14 views

R: Plot trees from h2o.randomForest() and h2o.gbm() [migrated]

Looking for an efficient way to plot trees in rstudio, H2O's Flow or in local html page from h2o's RF and GBM models similar to the one in the image in link below. Specifically, how do you plot trees ...
0
votes
1answer
13 views

Out of Bag Prediction Error Estimate from Random Forest Regression (i.e., not classification) [on hold]

I would like to use the OOB cases from a random forest fit to estimate the mean squared prediction error so I don't have to cross-validate. I am using the randomForest package in R. It is clear from ...
0
votes
0answers
9 views

How can one quantify the variable importance dilution effect in random forests (and similar statistical learning methods)?

In Applied Predictive Modelling (Kuhn, Johnson, 2013, p 202), the authors refer to a dilution effect whereby compared to a single tree or a classical regression technique, the difference in importance ...
0
votes
0answers
14 views

Is it possible that my 200 boostrap samples have better performance that my original data? [closed]

When I check the performance of 200 bootstrap samples, all of them have better scores than my original data. (I am not asking about the learning technique because it is a conceptual question, but it ...
0
votes
0answers
14 views

Create a predictive user model

I am a bit lost with creating a user model in R. I would like to create a model that predicts whether a user is likely to do an action or not, based on data on his past behaviour (Target variable ...
2
votes
1answer
47 views

In random forest, what happens if I add features that are correlated?

(Sorry for the potentially unclear English, not a native speaker) I'm training a random forest, trying to predict market shares of future stores on geographical areas. I have many features for these ...
0
votes
0answers
10 views

converting feature from string to categorical reduces classification accuracy

I am working on San Francisco crime classification problem from kaggle. https://www.kaggle.com/c/sf-crime during the work I encountered something unexpected. I applied scikit learn's random forest ...
0
votes
1answer
34 views

Random Forest model good train and test performance but bad “real world” performance

I am working on a classification problem where I need to classify objects based on a visual data. There are a couple hundred different classifications to be made and I have around a million plus ...
0
votes
0answers
35 views

How to know the importance of all the variables and levels in them using Random Uniform Forest in R?

I have a dataset containing 3 parameters (Region( factors - say US,UK,Aus,NZ),Domain or Industry( factors - say IT,Electrical,Mechanical) and Scope - good or bad). Using Random Uniform Forest package ...
0
votes
1answer
24 views

The meaning of Classification Accuracy

I'm working on San Francisco Crime dataset, and only get about 20% classification accuracy. I used Random Forest Method. So how I can Interpret the result? I did EDA firstly, but how can I use EDA to ...
2
votes
0answers
72 views

Random forest for forecasting univariate time series

I read few articles on random forest and its implementation in various fields. But I hardly found any literature on its implementation on forecasting univariate time series. Can it be used for ...
0
votes
0answers
20 views

Difference between rulefit and random forest

I'm trying to understand the difference between these a bit better. I understand pretty well how random forests work but I guess I'm more hazy on rulefit and how exactly it's different. I know rulefit ...
0
votes
0answers
4 views

Weights in adaboost/decision tree(cart)

I'm trying to implement adaboost using decision trees. But I'm confused over the weights. I am unable to understand how to incorporate weights in training process, how the formulas for node entropy ...
0
votes
0answers
6 views

Gini impurity and generalization error

Has anyone seen papers on relationship between information-based criterions (such as Gini impurity, information gain etc.) and generalization error? Is there theoretical justification of using such ...
0
votes
0answers
12 views

In Random Forest, why IncNodePurity is biased?

I've seen this statement many times, however, I could not find an explicit demonstration of why IncNodePurity biased (actually, how does one define theoretical value of importance, is not so clear, ...
0
votes
1answer
22 views

Do Random Forests use boosting

Ok so I think I have listened to a few wrong discussions on random forests because now I have a very confused question. With respect to Random Forests and bagging/bootstrapping, I'm good there. The ...
1
vote
0answers
28 views

Tuning Parameters for Boosting/Bagging/Random Forest

I want to use tree-based classifiers for my classifiaction problem. I'm thinking about bagging, boosting (AdaBoost, LogitBoost, RUSBoost) and Random Forest but I'm unsure about the tuning parameters, ...
3
votes
2answers
58 views

Out of Bag Error makes CV unnecessary in Random Forests?

I am fairly new to random forests. In the past, I have always compared the accuracy of fit vs test against fit vs train to detect any overfitting. But I just read here that: "In random forests, ...
0
votes
0answers
24 views

How to select a model from leave one out cross-validation

I have a set of 400 positive vectors and hundreds of millions of negative ones. I have split the data into a training and test set each of 200 positive vectors and lots of negative ones. I would ...
1
vote
2answers
35 views

Can trees or random forests learn ratios

This is a question about feature engineering for decision trees/random forests. Given two continuous variables X1 and X2, is it ...
1
vote
0answers
43 views

How to handle over-prediction in Random-Forest

I have a regression Random Forest model (generated using H2O and R). After tuning and building the model, I plot the predicted value vs. the labeled value of both the train and test datasets. In a ...
1
vote
0answers
18 views

Preprocessing Random Forest With Lots of Features

I'm working on a project for uni where I have to predict a two-class problem, related to acceptance (or not) of a patent demand. Initially, I have a dataset separated into training and test data. My ...
0
votes
0answers
12 views

Weka Experimenter Tool (xx/yy/zz) explanation

I am using Weka experimenter tool and I need help to fully understand how this count works. I found this explanation in a paper: The annotation v or * indicates that a specific result is ...
0
votes
0answers
15 views

Feature importance RF

What is the difference between 'DeltaCriterionDecisionSplit' in the Treebagger function and predictorImportance() function from tree ensemble in matlab? Thanks.
0
votes
0answers
8 views

R randomForest importance=FALSE behavior

From the R help for the values of a randomForest object: "importance - ...(if importance argument=TRUE) The last column is the mean decrease in Gini index ... . If importance=FALSE, the last measure ...
3
votes
1answer
74 views

Is there a method to plot the output of a random forest in R?

Nice and simple. I've spent two hours googling, reading cross validated, and several r blogs to attempt to find a simple method of outputting the representative tree in R. I was attempting to ...
1
vote
1answer
21 views

Drop in results upon addition of new features in random forest model

I am training a classification random forest for object detection in images. I have several features (like HoG, edge features etc) which work good enough separately. But when I train using all ...
1
vote
0answers
31 views

Modelling house energy production using month as a variable

I'm attempting to model the energy production of a set of houses for which data on temperature and daylight over 22 months is available. The data is arranged such as such: ...
1
vote
1answer
12 views

How to extract a splitting point for numerical values in a random forest model?

I have a dataset including discrete and continuous variables on which I ran a random forest model using the r-package randomForest. I have read several times that ...
2
votes
1answer
42 views

Find max value of random forest regressor output

I was wondering, for scikit learns regressors (extra trees, random forest regressor etc), how can i find the combination of inputs that would give me the max value of the target variable? Other than ...
0
votes
1answer
43 views

Interpretation of Random Forests [duplicate]

I have a question regarding interpretation of results of a random forest that I created. First, some background regarding the data: I have a dataset that consists of 100 true instances and 1000 ...
1
vote
0answers
34 views

How do I add cross validation for a random forest regression?

The error percentage of regression changes with change in the train and test data which I am deciding randomly. Cross validation can overcome this but how do I apply it for my regression model?
1
vote
0answers
13 views

Checking noisy data from random forest

I am running a randomforest on some unbalanced data with five classes (different behaviours). I have downsampled my data to address the balance issue but I believe there is a substantial amount of ...
2
votes
0answers
28 views

how to model longitudinal big data?

Traditionally we use mixed model to model longitudinal data, i.e. data like: ...
0
votes
1answer
26 views

Random Forests with modified partitioning criteria

Here is the context of my question : I'm doing binary classification with unbalanced classes. The measure of performance I'd like to maximise is a modified F-measure : $$ F_{\alpha} = ...
0
votes
1answer
17 views

random forest for density estimation using sklearn [closed]

I want to use or extend sklearn-randomForest for density estimation. I don't know to tackle it. I read A. Criminisi and his team work on random forest as a unified framework where they first ...
4
votes
3answers
374 views

random forest - summarize two features in one without losing information

I am training a random forest on a dataset including both categorical and numerical features. In particular I have a binary feature, call it $x_1$, which has $0$ or $1$ as possible outcomes. I also ...
0
votes
0answers
30 views

Difference in results for predict on caret package “train” object and “train$finalModel” object

Newish to R and new to CrossValidated. I have a question about the predict method for caret "train" objects. I'm running a randomForest model using caret package and am trying to produce some simple ...
1
vote
1answer
52 views

Feature subsampling with gradient boosting

A key component in building random forest models is feature subsampling, i.e., building each individual tree with only a percentage of predictors chosen randomly by tree. The literature often ...
0
votes
0answers
20 views

Difference of feature importance from Random Forest and Regularized Logistic Regression

I have 13 features in a classification task and I use Random Forest, L1 logistic regression and L2 logistic regression for as separate classifiers and would like to compare their performance. Although ...
0
votes
0answers
23 views

Optimizing a model for a limited budget

I am building models to predict probability of failures against a list of approximately 500K assets. I want to optimize my models for maximum predictive performance on a fixed (limited) number of ...
0
votes
0answers
15 views

testing AUC greater than training AUC? [duplicate]

I have about 30,000 samples with around 500 features. I randomly selected 10% as training dataset and another 10% as test1 and the remaining 80% as test2. I used randomForest to build model using ...
2
votes
1answer
16 views

Is it necessary to use warm_start when tracking oob_score in scikit RandomForestClassifier?

I'm planning on doing feature-selection with RandomForestClassifier by using the feature_importances and ...