I have an Excel model which predicts the number of customers for a given month. The prediction depends on a churn rate. I have the absolute error (actual vs predicted), along with squared error and sum of square error.

My question is:

Would it better to find a churn rate that minimizes the absolute for each period (year, month) or find a churn rate that minimizes the sum of squared errors? Does the former even make sense to do?

enter image description here

  • 1
    $\begingroup$ This depends on your loss function, I think. $\endgroup$ – Richard Hardy Nov 28 '18 at 19:07

If you minimise absolute error, you are implicitly assuming that your errors are Laplacian distributed and if you minimise mean-squared error, you're implicitly assuming they're normally distributed.

Due you have any reason a-priori to believe one of the other? Are your tails long? Pedantically, users are discrete, so are either appropriate ? If the numbers are quite large, continuous isn't a bad approximation but if your numbers are small, you might consider a discrete distribution leading to an entirely different distribution.

A-priori arguments aside, train a model using least squares, and then plot a histogram of your prediction errors $y_{i} - \hat{y}_{i}$. Does this distribution look normal? Train the model using absolute error and plot $y_{i} - \hat{y}_{i}$, does this distribution look Laplacian?

  • $\begingroup$ The conclusions you draw about "implicitly" themselves make the implicit assumption that you are using maximum likelihood. As @Richard Hardy suggests in a comment to the question, it's often more appropriate to consider the loss function rather than exploring distributional assumptions. $\endgroup$ – whuber Nov 28 '18 at 20:23
  • $\begingroup$ yes, that's fair. Would your default approach not be to use max-likelihood or max posterior unless you had a good reason to use something else? $\endgroup$ – gazza89 Nov 28 '18 at 20:33
  • $\begingroup$ My default is to explore the loss with the client in an effort to gather information relevant to deciding on an appropriate procedure. $\endgroup$ – whuber Nov 28 '18 at 20:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.