Can I learn binary classification and linear regression in the same network?

Question

I would like to train a neural network on an input signal and have it learn several unrelated decisions simultaneously (performing binary classification, one-class classification, and linear regression) to tell me different things about the input signal which I care about.

Note that this isn't something like multi-class classification, since I'll be using several different objective functions. Would simply summing my objective functions make the network less likely to converge, or take much longer to train? I will probably have to worry about the relative weights of each objective function. Is there any existing work that shows this is a bad idea? Should I just train several different classifiers independently on the same input signal? My motivation for using a single network is to reduce computation during inference.

Answer 1

All a neural network does is minimize some loss function. A sum (or multiple) of loss functions is a perfectly fine loss function in its own right.

In fact, this is quite a common setup. L(whatever) regularization is an extra term in the loss function, for one example. A VAE doesn't even work properly unless you use a sum of two losses, for another.

If neither loss term dominates the total, this can actually improve your network - e.g. overfitting on one task may increase the loss on another, so the training algorithm has a reason to look for a better solution.

As for the practical effects - depends. You're adding extra parameters, so a single pass is necessarily more expensive just due to the extra operations required, but you only have to train one model instead of three. Convergence depends on a lot of factors, so there's no general answer.

They may converge faster by avoiding bad minima of the individual tasks, or slower/worse if, say, you don't have enough capacity to solve all the tasks at once properly.