# What is the difference between Multitask and Multiclass learning

Consider a image labeling problem, where I need to assign one or more labels to an image. The possible labels are human, moving , indoor. Human means there is a human in the picture, moving could mean whether the human is runing/walking etc, and indoor classifies whether the image is indoor or outdoor.

Now I can train a DNN with 3 nodes in the output layer using labeled training data. (say node 1, is for +/- human, node 2 +/- moving and so on)

I always thought that this is just another way of doing multi-class classification using DNNs. (Another alternative for multi-class training would have been to train many DNNs with one output node in a pairwise or 1-vs-all fashion.)

But it seems like what I described (training a DNN with 3 output nodes) is Multi-task learning. ( I read wiki on MTL and looked at these papers)

My question is: is the scenario I described MTL? If not what am I missing? If yes, is there anything more to MTL? (clearly there is)

Also I notice in MTL a image can have multiple labels, but usually in multiclass an image is assigned only one - is this the difference?

Just to give a more clear understanding, I have explained each terminology with examples

• Multiclass classification/(One-Vs-One and One-Vs-ALL):
• Multilabel classification:
• This Classification task assigns a set of target labels to each sample.

• E.g. Building a classifier for a self-driving car that would need to detect several different things such as pedestrians, detect other cars, detect stop signs in an image!.

• E.g. A document could have multiple topics!

• Loss function used would be "logLoss"

• Multioutput regression/ Multitarget regression/multidimensional linear regression:
• This task assigns a sample to a set of target values!

• E.g.predicting several properties for each data-point, such as wind direction and magnitude at a certain location

• Last layer activation should be linear

• Loss function used would be "MSE"

Now let's comes to the difference between multi-task learning(one subset is a multilabel classification or multioutput regression) and multiclass classification problem! :

• Multi-class classification: You are assigning a single label (could be multiple labels such as MNIST problem) to the input image as explained above.

• Multi-task learning (one subset of it is multi-label classification) You're asking for each picture, does it have a pedestrian, or a car a stop-sign or traffic-light, and multiple objects could appear in the same image (one image can have multiple labels!).

• In other words, If you train a neural network to minimize this cost function (Log loss) you are carrying out multi-task learning because what you're doing is building a single neural network that is looking at each image and basically solving four different classification problems. It's trying to tell you does each image have each of these four objects in it?.

• Or you could achieve it by just training four separate neural networks instead of train one network to do four things!

• Note: But if some of the earlier features in the neural network can be shared between these different types of objects, then you find that training one NN to do four things, results in better performance than training four completely separate neural networks to do the four tasks separately. That's the power of Multi-task learning!

To put everything into perspective, Multi-task learning will make more sense when you have the below requirements:

• Training on a set of tasks that could benefit from having shared lower-level features!

• If the amount of data you have for each task is quite similar!

@RockTheStar, I think your transfer learning suggestion isn't correct! What you are trying to say & have similar(but weak) meaning for transfer learning, I will try to explain with the help of an example. Suppose you have 100 different tasks and you have 1000 train examples for each! If you build you separate NN for classifying each class you won't do much good! due to a limited number of training example (1000) but if you build a single network for solving 100 different, let's say classification problem then, you have (1000*1000)(Million instead of just 1000 for each separate task) training examples that can provide some knowledge that helps every other task among these 100 tasks. This is a big boost!

• Can train a big enough NN to do well on all the tasks!

• Very nice analogy. What activation do I use at output layer for a problem where I have to detect the presence of pedestrian or traffic sign or traffic light? My images can have only one or all or multiple things? Jun 19 '20 at 5:55
• @NaveenKumar, logloss, imgs will always have many objects eg. cars, pedestrians, road signs, etc! Your annotations of such objects within the train & val datasets guide N/W to learn about them! If you don't provide proper annotations, the N/w will never be known what it's looking for in an image! By showing the same object annotated at multiple images(train-set) at a diff location, color & lighting conditions you let the n/w learn the representation of that particular object & then give the n/w an image to make predictions to say from n/w learned to understand (repr.) what this object is!
– Anu
Jul 10 '20 at 16:00

Slight correction on @RockTheStar answer. Multi-task learning is not when you learn for one task and then transfer to another as was suggested, instead the tasks are learned in parallel similar to the usual multi-class classification setup.

I suppose the simple distinction that can be made is that the outputs are not necessarily classes of the same of a single task, but two or more loosely related tasks that are sharing information.

Multi-class learning: have multiple class labels that you want to classify. For example, I have labels cat, dog, and pig. These are animals. I want to make a DNN to classify them. That's a multi-class learning.

Multi-task learning: is somewhat like transfer learning. Basically, you create one model for one classification task and them use it in another classification task (after modification).

In you case, you can create a two-label DNN to classify if the image is human or not. Then use that model parameters parameters to start to build another classifier that classify moving or not. I will consider your case more like multi-task learning because the subjects of interests are different.

• So in my case I am doing MTL in parallel? Is there any reason to do it step-by-step like you describe? If I understand correctly i first train a DNN for +/-human (single output node) then take that network and train over it with output label changed to say +/-moving.
– A.D
Jul 15 '15 at 17:34
• Human and moving are two different subjects. The features detectors you obtain from human DNN is very likely to differently to moving. So, in order to, let say, classify whether an input is moving, what you need to do is first pass identify whether the input is human first, and then if yes, put the input to the moving network and classify if the input is moving or not. In this progressing step, you need to build two network. One with network trained with data that is human and non-human. Another network trained with data this (already human) moving or non-moving. Jul 15 '15 at 20:30
• "what you need to do is first pass identify whether the input is human first, and then if yes" this seems the opposite of what MTL is - i'm still unclear. What you are saying sounds like cascading of classifiers. Consider an alternate task, +/-human, +/-animal, +/-indoor.
– A.D
Jul 16 '15 at 16:07
• "Multi-task modeling basically means you use one constructed model from one task to another task. " In your case, multi-task may not work. Because the network designed to classify human or not I don't think is too related to classify moving or not. So a cascading of classifiers may be better. Give an example on multi-task modeling. Let's say you train a model to classify panda or not. Since panda look like human (4 limbs, can sit, etc), you can then retrain the model (initialized with the parameters obtained from panda) to classify human or not. Jul 16 '15 at 18:28
• Multi-task learning must not necessarily use transfer learning. You can train the network on all tasks simultaneously. The idea is that features necessary for one task can also be useful for the other tasks. May 7 '20 at 9:54

I notice in MTL a image can have multiple labels, but usually in multiclass an image is assigned only one - is this the difference?

Yes.

In multi-task learning, you can detect multiple objects (i.e. 'car', 'human', 'cat', 'tree') in one given image. If there exist both a human and a car in your model, Your labels are [1,1,0,0] (from ['car', 'human', 'cat', 'tree']).

In multiclass classification, you will have only one gold (1) in your labels, [1,0,0,0]`.

There is quite some information in all of the answers provided thus far, but maybe a summary answer might be useful for people with the same / a similar question:

• Multi-class learning is the terminology used when you predict a single label for each input, but each label is a single element from a set of possible labels. Taking the example from this answer: you have an image of an animal and the goal is to label whether it is a cat OR a dog OR a pig. It can only be one of these animals, but there are multiple options. To train a network on this kind of problems, you would generally use the Cross-entropy loss function.

• Multi-task learning is when you have different problems that need to be solved simultaneously (as stated in this answer. Each problem could be a binary classification problem (as the OP example) or another multi-class problem. Taking the example of the animal images again: a second task could be to predict whether the environment on the picture is sunny OR cloudy OR indoor. So now the goal is to find the animal label AND the environment label. The key idea is that solving one task could help solving the second task (in this example: cats are more likely to appear indoors than pigs). For training a network in this case, each output can have its own loss function. The loss functions can be combined by summing them up, using weights to balance the importance of each task.

• Multi-label learning (as mentioned in this answer) can be considered a special case of multi-task learning. In this case, there could be one or more animals in the image and you want a label for every animal in the image. In practice this is solved by breaking down this task in multiple binary problems. If the possible labels would be dog, cat and pig, the binary problems would be (1) Is there a dog in the image, (2) Is there a cat in the image, (3) is there a pig in the image. These sub-problems can then be tackled by multi-task methods.

Note that I ignored regression as a whole, but multi-task learning could also combine classification and regression problems.