12
$\begingroup$

Can a Random Forest be trained to appropriately predict count data? How would this proceed? I have quite a extensive range of values so classification doesn't really make sense. If I would use regression would I simply truncate the results? I'm quite lost here. Any ideas?

$\endgroup$
  • 1
    $\begingroup$ can't you use poisson regression? $\endgroup$ – RJ- Feb 25 '13 at 5:54
  • $\begingroup$ I wanted to use something non-parametric. Don't really remember poisson regressions assumptions but I'm pretty sure one of them is that the observations are independant and that is deeply not fulfilled here. Could this affect me much? $\endgroup$ – JEquihua Feb 25 '13 at 13:44
  • 2
    $\begingroup$ Have you simply tried making RF regression (possibly also in log)? It may just work good enough. $\endgroup$ – user88 Feb 25 '13 at 17:39
  • 1
    $\begingroup$ Have not. But that was my first instict. Log or square root transformation. But I wanted to see if anyone had any experience on this. $\endgroup$ – JEquihua Feb 25 '13 at 20:23
  • $\begingroup$ I tried just doing regression on the response, the log(response) and sqrt(response) and got nothing good. I think the problem is more that my independant variables done explain the response. Oh well. $\endgroup$ – JEquihua Mar 3 '13 at 0:44
8
$\begingroup$

There is a R package called mobForest which can fit a real random forest for count data. It is based on mod() (model-based recursive partitioning) in the party package. It performs Poisson regression if the family argument is specified as poisson(). The package is no longer in the CRAN repository, but formerly available versions can be obtained from the archive.

If you are not restricted to random forest / bagging, a boosting version is also available for count data. That is, gbm (generalized boosted regression models). It can also fit a Poisson model.

$\endgroup$
4
$\begingroup$

I see a few possibilities.

  • You could bin the response into a few arbitrary categories and use a classification tree
  • If the counts are typically very low, 0, 0, 0, 1, 0, 3, 0, 2, you could treat each integer count as a class and again use a classification tree (probably not your case). In these cases, it's going to be harder to get a high variance explained type metric as opposed to continuous regression.
  • If the counts are not typically low and there is a lot of variation, I'd just go for it with a regression tree. Using poisson regression over linear regression, for instance, is only gravy when it comes to getting a good linear predictor. If you're not seeing good predictive power with the random forest, then I doubt a fancier model that specifically accommodates count data is going to do a lot for you.
$\endgroup$
3
$\begingroup$

Well, its not random forest, but CatBoost supports a poisson loss function that could be used for count regression with boosted trees:

https://tech.yandex.com/catboost/doc/dg/concepts/loss-functions-docpage/

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.