Is there any algorithm combining classification and regression? I'm wondering if there's any algorithm could do classification and regression at the same time. For example, I'd like to let the algorithm learn a classifier, and at the same time within each label, it also learns a continuous target. Thus, for each training example, it has a categorical label and a continuous value.
I could train a classifier first, and then train a regressor within each label, but I'm just thinking that if there's an algorithm that could do both, it would be wonderful.
 A: Multi-Task Learning MLT allows different types of loss-functions ( for example, least-square for regression and logistic or Hinge loss for classification) to be optimized simultaneously.
the components of this heterogeneous loss function can be weighted to control/ distinguish the main task from the secondary one. if the two tasks do not have the same learning difficulties and convergence rates; a stop criterion has to be introduced for the simpler task to avoid overfitting. a 3rd component can also be introduced to the loss-function to ensure smoothness of the whole learning process.
the heterogeneous loss function may look like that (a case for regression and classification):
notice the applied weight to the logistic loss function, and the last regularization term for eights penalization

Now if we want to implement this with Pytorch, we have to split the output and run it through different criteria (again MSE for regression and logistic loss for classification)
let yhat the inintial output of the model that is splited into yhat_1 and yhat_2 such :
yhat = concat(yhat_1, yhat_2)
the same for y the ground truth.
in the learning step the model should be optimized as follow:
criterion1 = nn.MSELoss()
criterion2 = nn.BCELoss()
loss1 = criterion1(yhat_1, y1)
loss2 = criterion1(yhat_2, y2)
loss = loss1 + lambda*loss2
loss.backward()

