# Using regression forest for a factor variable

I want to predict if a customer is interested in a new product and I use the randomForest package for that.

Target variable : factor (Yes or No) so I use the randomForest for classification :

randomForest(x=train,y=labels_train,xtest=test, ytest=labels_test,  ntree=100)


Variables :

• the city (A,B,C)
• Gender : Male / Female
• Age classes : 18-25, 25-59 ,60 and +

Problem : only 40% of the residents of the city A are interested (same proportion for both gender and all age classes).

In this case, ALL the tree of the forest assign "Not interested" for the residents of this city.

According to the forest these residents have a probability of 0% to be interested by the product.

Proposition : Using regression instead of classification. In this way, each tree will declare 40%, and the vote is replace by an average wich give a probability of 40%

Could you confirm it's correct to do that when the goal is not really to classify but to have a probability of interest, in order to only contact people who have a probability higher than 30% for example.

• Why are you using a random forest instead of a logistic regression? (The feature space is quite low dimensional.) Jan 13 '14 at 17:41
• It's just an example, I have dozen of variables in reality Jan 13 '14 at 17:52
• I would not do this. the classification randomforest should give you reasonable predicted probabilities. if it isn't, there is an issue with the data/model that should be clarified. Mar 15 '14 at 3:37
• For each (city,gender,age) triplet you have many different training examples? For instance, a part of you training set may look like (A,male,18-25,true),(A,male,18-25,false),(A,male,18-25,true),(A,male,18-25,true),(A,male,18-25,false) ? Sep 5 '14 at 21:13

Regression Random Forests have been used to estimate probabilities/risk for binary outcomes. See open-access article cited below. Apparently the key is to avoid end node purity - thus the nodes are limited in size to 10% of training sample size. The article is unfortunately poorly written (impressively so) and thus find it hard to have confidence in the methods suggested.

• Probability Machines: Consistent Probability Estimation Using Nonparametric Learning Machines. Malley JD et al in Methods Inf Med 2012

The regression forest should works, but if I remember, the trees will not be the same than in the classification forest (and I don't remember why)