Predicting if some variable is $\geq C$?

Say you're given a dataset where the response $$y$$ is continuous. The only prediction you're after is whether $$y \geq C$$, i.e., where the response is greater than some value.

In this case, would it be better to

(1) Use a regression algorithm that produces a continuous output, and then manually check if the response is $$\geq C$$?

(2) Use a classification algorithm. To do this, we must first preprocess the training data so the samples with $$y_i \geq C$$ are set to $$1$$ for binary classification and 0 otherwise.

It seems that both are feasible, but which is better, or what are the advantages/disadvantages of both?

• Regression will tell you not only if it is bigger than C but also how much bigger it is, this is an important piece of information. Jul 29 '20 at 5:42
• @user2974951. I'm assuming you don't care about how much bigger it is and just that it's bigger or not. In other words you're given continuous data, but the only thing you care about is a binary answer. Jul 29 '20 at 5:44

Interesting question. First of all, classic linear regression was developed for applications where the scatter is normally distributed. If you plot the residual distribution it should have the classic bell shape. When your data adhere to the these model prerequisites, your can just as well use linear regression. The confidence bounds for linear regression predictions are also known, it is advisable that you use these.

When your predictor variables come from discrete, or multimodal continuous distributions then you can benefit from a nonparametric classifier. For example the histogram classifier or the K-nearest neighbor classifier can be used. I presuppose that training a neural network for the prediction of $$y$$ would be somehow 'over-the-top'.

• For this types of distributions, the the prerequisites of linear regression are clearly not fulfilled. With a histogram classifier, a nonparametric criterion is used to determine the most likely outcome of $y$. Jul 29 '20 at 20:30