I used the package for random forest. It is not clear to me how to use the results. In logistic regression you can have an equation as an output, in standard tree some rules. If you receive a new dataset you can apply the equation on the new data and predict an outcome (like default/no default). Or saying the customers with characteristics a and characteristics b will have a default, so you can predict the outcome before it happens. That is the scoring tecnique.

Is it possible to use random forest in a similar situation, or how would you use the results of a RF?

my python code:

from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import roc_auc_score

#creating a test and train dataset

from sklearn.cross_validation import train_test_split

train, test = train_test_split(df, test_size = 0.3)

clf = RandomForestClassifier(max_depth = 30, min_samples_split=2, n_estimators = 200, random_state = 1)

#training the model
clf.fit(train[columns], train["churn"])

#testing the model
predictions = clf.predict(test[columns])


print(roc_auc_score(predictions, test["churn"]))
  • $\begingroup$ What do you mean results of a RF? RF's are methods to either classify or regress. Do you mean the variable importances? $\endgroup$ – O.rka Jul 7 '16 at 7:06
  • $\begingroup$ As an output of the model, in classifying, you have for each row in your dataset a binary variable (0-1) or the probabilities that the events occurs. Also you have a list of feature importance. what is the next step, what to do with this informations? $\endgroup$ – progster Jul 7 '16 at 7:11
  • $\begingroup$ Which packages are you using? randomforest ? $\endgroup$ – mkt - Reinstate Monica Jul 7 '16 at 7:33
  • $\begingroup$ I used Python, from sklearn.ensemble import RandomForestClassifier, but if you have any information feel free to talk also about R $\endgroup$ – progster Jul 7 '16 at 7:38
  • $\begingroup$ I've added an example illustrating how it is frequently used. Let us know if you have more specific questions. $\endgroup$ – mkt - Reinstate Monica Jul 7 '16 at 8:00

Once you train your random forest, one could use it to

  1. classify new data in the case of categorical variables, or predict quantitative values in the case of a continuous variable (prediction),

  2. quantify the relative contribution of different variables for separating groups (understanding), or

  3. quantify the partial effect of individual variables on group identity (understanding).

Note that the last can be nonlinear in multiple dimensions.

#Create example data with 2 groups (a & b), one variable 
#that differs between groups (m1), and the other randomly drawn (m2)    
dat <- data.frame(group = c(rep('a' , 50), rep('b', 50)), 
            m1 = c(sample(1:10, 50, replace = TRUE), 
                sample(5:20, 50, replace = TRUE)),
            m2 = c(sample(1:50, 100, replace = TRUE)))


#Train random forest 
rf <- randomForest(group ~ m1 + m2, data = dat,importance = TRUE, 
                   ntree = 1001)

#Create new data to be classified
newdats <- data.frame(m1 = sample(1:20, 10), m2 = sample(1:50, 10))

#1. Predict the group identities of the new data points
predict(rf, newdats)

#2. Examine which variables are useful in separating the 2 groups

#3. Plot the partial effect of each variable on the estimated group identity 
# (i.e. partial dependence plots)
partial(rf, pred.var = 'm1', plot = TRUE)
partial(rf, pred.var = 'm2', plot = TRUE)
  • $\begingroup$ I think that is more clear now, I don't have a target variable in newdats, you simply apply the trained model to new data, correct? if it's correct it was the situation I was looking for. On the other hand I guess that "varImpPlot(rf)" could give and idea of which variable is more important, but not the exact rule that explicate variable relationship (like an equation in logistic regression or tree rules) $\endgroup$ – progster Jul 7 '16 at 11:09
  • $\begingroup$ 1) I'm not exactly sure what you mean by 'target variable'. The new data must have any variables you trained the random forest on. 2) Exactly. You do not get an simple parametric relationship from the random forest. Each underlying tree has a sequence of rules that when aggregated across trees gives you a prediction. But because the hierarchy differs from tree to tree, I do not think that the rule cannot be easily simplified to an equation. $\endgroup$ – mkt - Reinstate Monica Jul 7 '16 at 11:41
  • $\begingroup$ for 'target variable' I mean the outcome you want to predict (like "churn" vs "not churn", like "y" in logistic regression equation, in your example should be "group", right? $\endgroup$ – progster Jul 7 '16 at 11:54
  • $\begingroup$ Ah, you're right, that is 'group' in my example. This prediction goal only makes sense if you do not have the 'target variable' in your new data. If you do, there's nothing to predict. $\endgroup$ – mkt - Reinstate Monica Jul 7 '16 at 12:01
  • $\begingroup$ @progster Does this address your questions? $\endgroup$ – mkt - Reinstate Monica Jul 8 '16 at 6:44

Random forests is or random decision forests is just an ensemble learning method for classification, regression and other tasks, which is constructed by a multiple of decision trees at training time to output the class or mean prediction of the individual trees.

would be easier if you post some of your implementation plus the data income and outcome

  • $\begingroup$ if you are familiar with Python I added in the post my code, just to confirm that is the correct approach. [columns] is a list of variables. I cannot share results sorry. Any improvement or "next step" tip is appreciated $\endgroup$ – progster Jul 7 '16 at 11:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.