17
$\begingroup$

Can a Random Forest be trained to appropriately predict count data? How would this proceed? I have quite a extensive range of values so classification doesn't really make sense. If I would use regression would I simply truncate the results?

I'm quite lost here. Any ideas?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ can't you use poisson regression? $\endgroup$
    – R-J
    Feb 25, 2013 at 5:54
  • $\begingroup$ I wanted to use something non-parametric. Don't really remember poisson regressions assumptions but I'm pretty sure one of them is that the observations are independant and that is deeply not fulfilled here. Could this affect me much? $\endgroup$
    – JEquihua
    Feb 25, 2013 at 13:44
  • 2
    $\begingroup$ Have you simply tried making RF regression (possibly also in log)? It may just work good enough. $\endgroup$
    – user88
    Feb 25, 2013 at 17:39
  • 1
    $\begingroup$ Have not. But that was my first instict. Log or square root transformation. But I wanted to see if anyone had any experience on this. $\endgroup$
    – JEquihua
    Feb 25, 2013 at 20:23
  • 1
    $\begingroup$ I tried just doing regression on the response, the log(response) and sqrt(response) and got nothing good. I think the problem is more that my independant variables done explain the response. Oh well. $\endgroup$
    – JEquihua
    Mar 3, 2013 at 0:44

3 Answers 3

Reset to default
10
$\begingroup$

There is a R package called mobForest which can fit a real random forest for count data. It is based on mod() (model-based recursive partitioning) in the party package. It performs Poisson regression if the family argument is specified as poisson(). The package is no longer in the CRAN repository, but formerly available versions can be obtained from the archive.

If you are not restricted to random forest / bagging, a boosting version is also available for count data. That is, gbm (generalized boosted regression models). It can also fit a Poisson model.

Improve this answer
$\endgroup$
7
$\begingroup$

I see a few possibilities.

  • You could bin the response into a few arbitrary categories and use a classification tree
  • If the counts are typically very low, 0, 0, 0, 1, 0, 3, 0, 2, you could treat each integer count as a class and again use a classification tree (probably not your case). In these cases, it's going to be harder to get a high variance explained type metric as opposed to continuous regression.
  • If the counts are not typically low and there is a lot of variation, I'd just go for it with a regression tree. Using poisson regression over linear regression, for instance, is only gravy when it comes to getting a good linear predictor. If you're not seeing good predictive power with the random forest, then I doubt a fancier model that specifically accommodates count data is going to do a lot for you.

Update (2020-12-11) Since writing this answer, a Kaggle contest (the M5 competition) showed me a situation where using Poisson loss in a LightGBM framework did really well for retail sales data with low counts. I don't know how much better it did than mean squared error but many of the public notebooks were using it, and it was an easy switch. I don't think my second bullet is a very good idea but I'll leave it up.

Improve this answer
$\endgroup$
4
$\begingroup$

Well, its not random forest, but CatBoost supports a poisson loss function that could be used for count regression with boosted trees:

https://tech.yandex.com/catboost/doc/dg/concepts/loss-functions-docpage/

Improve this answer
$\endgroup$
1
  • $\begingroup$ The tools keep coming! I updated my answer to say a similar thing about LightGBM, which people were using in the recent M5 competition with Poisson loss. $\endgroup$
    – Ben Ogorek
    Dec 11, 2020 at 14:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.