# What is the difference between Multitask and Multiclass learning

Consider a image labeling problem, where I need to assign one or more labels to an image. The possible labels are human, moving , indoor. Human means there is a human in the picture, moving could mean whether the human is runing/walking etc, and indoor classifies whether the image is indoor or outdoor.

Now I can train a DNN with 3 nodes in the output layer using labeled training data. (say node 1, is for +/- human, node 2 +/- moving and so on)

I always thought that this is just another way of doing multi-class classification using DNNs. (Another alternative for multi-class training would have been to train many DNNs with one output node in a pairwise or 1-vs-all fashion.)

But it seems like what I described (training a DNN with 3 output nodes) is Multi-task learning. ( I read wiki on MTL and looked at these papers)

My question is: is the scenario I described MTL? If not what am I missing? If yes, is there anything more to MTL? (clearly there is)

Also I notice in MTL a image can have multiple labels, but usually in multiclass an image is assigned only one - is this the difference?

Slight correction on @RockTheStar answer. Multi-task learning is not when you learn for one task and then transfer to another as was suggested, instead the tasks are learned in parallel similar to the usual multi-class classification setup.

I suppose the simple distinction that can be made is that the outputs are not necessarily classes of the same of a single task, but two or more loosely related tasks that are sharing information.

Just to give a more clear understanding, I have explained each terminology with examples

• Multiclass classification/(One-Vs-One and One-Vs-ALL):
• Multilabel classification:
• This Classification task assigns a set of target labels to each sample.

• E.g. Building a classifier for a self-driving car that would need to detect several different things such as pedestrians, detect other cars, detect stop signs in an image!.

• E.g. A document could have multiple topics!

• Loss function used would be "logLoss"

• Multioutput regression/ Multitarget regression/multidimensional linear regression:
• This task assigns a sample to a set of target values!

• E.g.predicting several properties for each data-point, such as wind direction and magnitude at a certain location

• Last layer activation should be linear

• Loss function used would be "MSE"

Now let's comes to the difference between multi-task learning(one subset is a multilabel classification or multioutput regression) and multiclass classification problem! :

• Multi-class classification: You are assigning a single label (could be multiple labels such as MNIST problem) to the input image as explained above.

• Multi-task learning (one subset of it is multi-label classification) You're asking for each picture, does it have a pedestrian, or a car a stop-sign or traffic-light, and multiple objects could appear in the same image (one image can have multiple labels!).

• In other words, If you train a neural network to minimize this cost function (Log loss) you are carrying out multi-task learning because what you're doing is building a single neural network that is looking at each image and basically solving four different classification problems. It's trying to tell you does each image have each of these four objects in it?.

• Or you could achieve it by just training four separate neural networks instead of train one network to do four things!

• Note: But if some of the earlier features in the neural network can be shared between these different types of objects, then you find that training one NN to do four things, results in better performance than training four completely separate neural networks to do the four tasks separately. That's the power of Multi-task learning!

To put everything into perspective, Multi-task learning will make more sense when you have the below requirements:

• Training on a set of tasks that could benefit from having shared lower-level features!

• If the amount of data you have for each task is quite similar!

@RockTheStar, I think your transfer learning suggestion isn't correct! What you are trying to say & have similar(but weak) meaning for transfer learning, I will try to explain with the help of an example. Suppose you have 100 different tasks and you have 1000 train examples for each! If you build you separate NN for classifying each class you won't do much good! due to a limited number of training example (1000) but if you build a single network for solving 100 different, let's say classification problem then, you have (1000*1000)(Million instead of just 1000 for each separate task) training examples that can provide some knowledge that helps every other task among these 100 tasks. This is a big boost!

• Can train a big enough NN to do well on all the tasks!

I notice in MTL a image can have multiple labels, but usually in multiclass an image is assigned only one - is this the difference?

Yes.

In multi-task learning, you can detect multiple objects (i.e. 'car', 'human', 'cat', 'tree') in one given image. If there exist both a human and a car in your model, Your labels are [1,1,0,0] (from ['car', 'human', 'cat', 'tree']).

In multiclass classification, you will have only one gold (1) in your labels, [1,0,0,0]`.

Multi-class learning: have multiple class labels that you want to classify. For example, I have labels cat, dog, and pig. These are animals. I want to make a DNN to classify them. That's a multi-class learning.

Multi-task learning: is somewhat like transfer learning. Basically, you create one model for one classification task and them use it in another classification task (after modification).

In you case, you can create a two-label DNN to classify if the image is human or not. Then use that model parameters parameters to start to build another classifier that classify moving or not. I will consider your case more like multi-task learning because the subjects of interests are different.

• So in my case I am doing MTL in parallel? Is there any reason to do it step-by-step like you describe? If I understand correctly i first train a DNN for +/-human (single output node) then take that network and train over it with output label changed to say +/-moving. – A.D Jul 15 '15 at 17:34
• Human and moving are two different subjects. The features detectors you obtain from human DNN is very likely to differently to moving. So, in order to, let say, classify whether an input is moving, what you need to do is first pass identify whether the input is human first, and then if yes, put the input to the moving network and classify if the input is moving or not. In this progressing step, you need to build two network. One with network trained with data that is human and non-human. Another network trained with data this (already human) moving or non-moving. – RockTheStar Jul 15 '15 at 20:30
• "what you need to do is first pass identify whether the input is human first, and then if yes" this seems the opposite of what MTL is - i'm still unclear. What you are saying sounds like cascading of classifiers. Consider an alternate task, +/-human, +/-animal, +/-indoor. – A.D Jul 16 '15 at 16:07
• "Multi-task modeling basically means you use one constructed model from one task to another task. " In your case, multi-task may not work. Because the network designed to classify human or not I don't think is too related to classify moving or not. So a cascading of classifiers may be better. Give an example on multi-task modeling. Let's say you train a model to classify panda or not. Since panda look like human (4 limbs, can sit, etc), you can then retrain the model (initialized with the parameters obtained from panda) to classify human or not. – RockTheStar Jul 16 '15 at 18:28