Transform a regression problem in a classification problem and express it ouput as probability function

Question

I am dealing with a regression problem related to predict the demand of a item by customers. This demand is an integer from 0 to 100 for example. I can employ SVR, Random Forest Regressor, etc. in order to predict future demand.

As far as I know, it is possible to discretize a regression problem and transform it in a classification problem, splitting it in ordered buckets (maybe 100 buckets in this case). I don't know what possible drawbacks could happen doing it (any link to a resource of this topic is welcome).

As a last step, I need express the result of the classifier as a discrete probability distribution. How can I go about doing this?

Answer 1

Have a look at Why should binning be avoided at all costs?

The question is what you want to predict for your problem setup? If you want to predict a value w/ uncertainty, then use adequate regression models. Nowadays you can use standard off the shelf methods for proper regression analysis without having to resort to 'binning' and using classification loss. See e.g. pyro.ai, XGBoostLSS, Tensorflow Probability, amongst other options.

Even when you choose to bin, you should use ordinal regression and not standard softmax regression loss / metrics; the latter is just penalizing the errors incorrectly . But then the question arises on why bother at all? If you measure continuous / float data, and your predicted targets are potentially real valued and useful that way, then why bother throw out information ( binning does that ) and work with approximate results.

Personally I have only seen this suggestion made when people are not familiar/ comfortable with regression metrics or algorithms; not because it's actually better at solving the prediction task.

Answer 2

Regarding binning and classifying vs predicting the actual values, this question discusses some advantages, and my answer addresses possible disadvantages. I stand by my comment there: It is...up to the scientist to evaluate the tradeoff between possible advantages and disadvantages.

If you do bin the values into categories and apply a "classification" model, such as a neural network or logistic regression, you automatically get the probabilities. These models might have a predict method that returns the most likely category, but they explicitly return values in the interval $[0,1]$ that can be interpreted and analyzed as probability values, such as with the predict_proba method in sklearn. A possible pitfall is that these predictions might not reflect the reality of how often events occur. For instance, predictions of $0.7$ should correspond to an event occurrence rate of $70\%$: in $70\%$ of the cases where an event probability of $0.7$ is predicted, the event should actually happen. There are attempts with varying levels of success at calibrating those values so they do reflect the true event occurrence probability.

For an SVM classifier in particular, you can get at the proability values using isotonic regression or Platt scaling, at least in the binary case. Extensions likely exist for multi-class cases.