I am modelling the number of pages generated by users on my homepage.

I tried a couple of models using Poisson and Negative Binomial distributions, with randomForest (mobForest).

However, I am unsure about the appropriate way of measuring the accuracy of my model on the test set. Are root mean square error (RMSE) and root mean square logarithmic error (RMLSE) proper measures here?

If not, what is the right way of doing that?

  • 1
    $\begingroup$ If your question has been answered to your satisfaction, you can accept an answer by clicking the check mark under the voting arrows. $\endgroup$ – Kodiologist Jul 18 '17 at 16:27

You can use proper scoring rules that look at the whole distribution of probabilities for your count variable. The choice of the most appropriate (proper) scoring rule in your case will depend on your objectives.


RMSE as well as mean absolute error seem fine for comparing observed to predicted counts. I don't see the motivation for RMSLE.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.