1
$\begingroup$

I want to predict if a customer is interested in a new product and I use the randomForest package for that.

Target variable : factor (Yes or No) so I use the randomForest for classification :

randomForest(x=train,y=labels_train,xtest=test, ytest=labels_test,  ntree=100)

Variables :

  • the city (A,B,C)
  • Gender : Male / Female
  • Age classes : 18-25, 25-59 ,60 and +

Problem : only 40% of the residents of the city A are interested (same proportion for both gender and all age classes).

In this case, ALL the tree of the forest assign "Not interested" for the residents of this city.

According to the forest these residents have a probability of 0% to be interested by the product.

Proposition : Using regression instead of classification. In this way, each tree will declare 40%, and the vote is replace by an average wich give a probability of 40%

Could you confirm it's correct to do that when the goal is not really to classify but to have a probability of interest, in order to only contact people who have a probability higher than 30% for example.

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ Why are you using a random forest instead of a logistic regression? (The feature space is quite low dimensional.) $\endgroup$
    – Michael M
    Commented Jan 13, 2014 at 17:41
  • $\begingroup$ It's just an example, I have dozen of variables in reality $\endgroup$
    – user3178882
    Commented Jan 13, 2014 at 17:52
  • $\begingroup$ I would not do this. the classification randomforest should give you reasonable predicted probabilities. if it isn't, there is an issue with the data/model that should be clarified. $\endgroup$
    – charles
    Commented Mar 15, 2014 at 3:37
  • 1
    $\begingroup$ For each (city,gender,age) triplet you have many different training examples? For instance, a part of you training set may look like (A,male,18-25,true),(A,male,18-25,false),(A,male,18-25,true),(A,male,18-25,true),(A,male,18-25,false) ? $\endgroup$
    – user31264
    Commented Sep 5, 2014 at 21:13

2 Answers 2

Reset to default
1
$\begingroup$

Regression Random Forests have been used to estimate probabilities/risk for binary outcomes. See open-access article cited below. Apparently the key is to avoid end node purity - thus the nodes are limited in size to 10% of training sample size. The article is unfortunately poorly written (impressively so) and thus find it hard to have confidence in the methods suggested.

  • Probability Machines: Consistent Probability Estimation Using Nonparametric Learning Machines. Malley JD et al in Methods Inf Med 2012
Improve this answer
$\endgroup$
0
$\begingroup$

The regression forest should works, but if I remember, the trees will not be the same than in the classification forest (and I don't remember why)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.