113
$\begingroup$

What is the difference between a multiclass problem and a multilabel problem?

$\endgroup$
1
  • 3
    $\begingroup$ I learned this concept and build my understanding with this post, they explained multi-label classification in a very elegant way. $\endgroup$
    – user235077
    Jan 23, 2019 at 14:47

7 Answers 7

95
$\begingroup$

I suspect the difference is that in multi-class problems the classes are mutually exclusive, whereas for multi-label problems each label represents a different classification task, but the tasks are somehow related (so there is a benefit in tackling them together rather than separately). For example, in the famous leptograspus crabs dataset there are examples of males and females of two colour forms of crab. You could approach this as a multi-class problem with four classes (male-blue, female-blue, male-orange, female-orange) or as a multi-label problem, where one label would be male/female and the other blue/orange. Essentially in multi-label problems a pattern can belong to more than one class.

$\endgroup$
6
  • 1
    $\begingroup$ @Dirkran Thanks for your explanation. Do you know any other source where i can get multilabel dataset other than csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/multilabel.html and mulan.sourceforge.net/datasets.html $\endgroup$
    – Learner
    Jun 13, 2011 at 10:42
  • 1
    $\begingroup$ @Learner sorry it isn't something I have worked on much. You might want to have a look at multi-task learning, which has some similarities to multi-label learning. Perhaps some of the datasets used for that might also be useful as benchmarks for mult-label learning. $\endgroup$ Jun 13, 2011 at 11:10
  • 1
    $\begingroup$ @DikranMarsupial Can you provide a reference for the definitions you provide? $\endgroup$
    – mareoraft
    Jan 22, 2021 at 16:49
  • 2
    $\begingroup$ @Mareoraft there is the wikipedia page en.wikipedia.org/wiki/Multi-label_classification althought I don't completely agree that it is a model that maps inputs x onto binary outputs y (as it may be that some of the labels are categorical rather than binary) $\endgroup$ Jan 22, 2021 at 17:36
  • 1
    $\begingroup$ @NateAnderson Yes, I think so. I am not sure why SciKit is making this distinction between "Multilabel classification" and "multiclass-multioutput" classification given that both are multilabel. The latter term seems a bit redundant as ordinary softmax is multiclass-multioutput but isn't multilabel. However neural networks and ML is full of these sorts of terminological problems and it is best not to get too focussed on definitions as they don't always have much top-down sense. $\endgroup$ Jun 24, 2023 at 5:55
84
$\begingroup$

Multiclass classification means a classification task with more than two classes; e.g., classify a set of images of fruits which may be oranges, apples, or pears. Multiclass classification makes the assumption that each sample is assigned to one and only one label: a fruit can be either an apple or a pear but not both at the same time.

Multilabel classification assigns to each sample a set of target labels. This can be thought of as predicting properties of a data-point that are not mutually exclusive, such as topics that are relevant for a document. A text might be about any of religion, politics, finance or education at the same time or none of these.

Taken from http://scikit-learn.org/stable/modules/multiclass.html


Edit1 (Sept 2020): For those who prefer contrasts of terms for a better understanding, look at these contrasts:

  • Multi-class vs Binary-class is the question of the number of classes your classifier is modeling. In theory, a binary classifier is much simpler than multi-class problem, so it's useful to make this distinction. For example, Support Vector Machines (SVMs) can trivially learn a hyperplane to separate two classes, but 3 or more classes make the classification problem much more complicated. In the neural networks, we commonly use Sigmoid for binary, but Softmax for multi-class as the last layer of the model.

  • Multi-label vs Single-Label is the question of how many classes any object or example can belong to. In the neural networks, if we need single label, we use a single Softmax layer as the last layer, thus learning a single probability distribution that spans across all classes. If we need multi-label classification, we use multiple Sigmoids on the last layer, thus learning separate distribution for each class.

Remarks: we combine multilabel with multiclass, in fact, it is safe to assume that all multi-label are multi-class classifiers. When we have a binary classifier (say positive v/s negative classes), we wouldn't usually assign both labels or no-label at the same time! We usually convert such scenarios to a multi-class classifier where classes are one of {positive, negative, both, none}. Hence multi-label AND binary classifier is not practical, and it is safe to assume all multilabel are multiclass.

On the other side, not all Multi-class classifiers are multi-label classifiers and we shouldn't assume it unless explicitly stated.


EDIT 2: Venn diagram for my remarks

Classifier

$\endgroup$
4
  • $\begingroup$ It's very very rare that you'd have two classes and want to assign both labels at the same time. What do you mean by that? Do you mean for instance a class is white dog, both white and dog in a single class, and the other is green cat? $\endgroup$ Oct 6, 2020 at 12:33
  • $\begingroup$ Updated my answer to improving the clarity. If you have a binary classifier, you have 2 classes. Say, DOG and CAT. You will assign one of those two classes, i.e. either DOG or CAT, but not both, or none to the same example. The implicit assumption of a binary classifier is that you are choosing one and only one class out of the available two classes. When you have multi-label classifier, the implicit assumption is that you have more than 2 classes. $\endgroup$ Oct 7, 2020 at 5:12
  • $\begingroup$ Thank you for that. $\endgroup$ Oct 8, 2020 at 9:10
  • 1
    $\begingroup$ Very Good answer for me. $\endgroup$
    – abdoulsn
    Oct 17, 2020 at 21:30
33
$\begingroup$

To complement the other answers, here are some figures. One row = the expected output for one sample.

Multiclass

One column = one class (one-hot encoding)

enter image description here

Multilabel

One column = one class

enter image description here


You see that:

  • in the multilabel case, one sample might be assigned more than one class.
  • in the multiclass case, there are more than 2 classes in total.

As a side note, nothing prevents you from having a multioutput-multiclass classification problem, e.g.:

enter image description here

$\endgroup$
2
  • $\begingroup$ Thanks for the visuals, very helpful! How can I interpret one of the rows in the multioutput-multiclass problem screenshot? Like maybe each column represents a different topic ("Sports" "Religion" "Politics"), and the value represents the "strength" of that? Doesn't that become a "multioutput regression" problem? In other words, what's an example of a multioutput-multiclass problem? (I guess scikitlearn calls them multiclass-multioutput, same thing?) $\endgroup$ Jun 21, 2023 at 23:52
  • $\begingroup$ An example of multiclass-multioutput described on scikit-learn: "For example, classification of the properties “type of fruit” and “colour” for a set of images of fruit. The property “type of fruit” has the possible classes: “apple”, “pear” and “orange”. The property “colour” has the possible classes: “green”, “red”, “yellow” and “orange” I was confused numbers in your third screenshot; but they are integers, I can imagine each integer maps to a label+class like "colour:green" or "fruit:pear" $\endgroup$ Jun 23, 2023 at 23:39
12
$\begingroup$

A multi-class problem has the assignment of instances to one of a finite, mutually-exclusive collection of classes. As in the example already given of crabs (from @Dikran): male-blue, female-blue, male-orange, female-orange. Each of these is exclusive of the others and taken together they are comprehensive.

One form of a multi-label problem is to divide these into two labels, sex and color; where sex can be male or female, and color can be blue or orange. But note that this is a special case of the multi-label problem as every instance will get every label (that is every crab has both a sex and a color).

Multi-label problems also include other cases that allow for a variable number of labels to be assigned to each instance. For instance, an article in a newspaper or wire service may be assigned to the categories NEWS, POLITICS, SPORTS, MEDICINE, etc. One story about an important sporting event would get an assignment of the label SPORTS; while another, involving political tensions that are revealed by a particular sporting event, might get both the labels SPORTS and POLITICS. Where I am, in the US, the results of the Superbowl are labeled both SPORTS and NEWS given the societal impact of the event.

Note that this form of labeling, with variable numbers of labels, can be recast into a form similar to the example with the crabs; except that every label is treated as LABEL-X or not-LABEL-X. But not all methods require this recasting.

$\endgroup$
4
$\begingroup$

And one more difference lies in that the multi-label problem requires the model to learn the correlation between the different classes, but in multiclass problems different classes are independent of each other.

$\endgroup$
2
$\begingroup$

All existing answers are great. I just wanted to crosslink scikit-learn because ...

  • crosslinking is good, and maybe some folks searching for this question use the scikit-learn or similar software packages
  • I started on the scikit-learn (the first Google result for the question), but I didn't read far enough to see that SciKit webpage does an excellent job explaining...
  • I resummarize the scikit-learn with examples of each type of problem, useful even if you don't use the scikit-learn
  • try to show how the scikit-learn's formats & language relate to the other existing answers (people use different words to mean the same thing)
  • The scikit-learn extends beyond just "multiclass" vs "multilabel"; it has "multiclass-multioutput" and "multioutput regression"

This table shows how the scikit-learn separates different types of problems:

[Problem Type] Number of targets Target cardinality Valid type_of_target
Multiclass classification 1 >2 'multiclass'
Multilabel classification >1 2 (0 or 1) 'multilabel-indicator'
Multiclass-multioutput classification >1 >2 'multiclass-multioutput'
Multioutput regression >1 Continuous 'continuous-multioutput'

Note the difference between Multiclass classification and Multilabel classification:

  • Multiclass has one target feature, and more-than-2 (mutually exclusive) outcomes
  • Multilabel has multiple target features, but each has a binary outcome (0 or 1)

Let's see some examples:

Multiclass classification

One target feature and more than 2 possible outcomes

For example, imagine you analyze photos and classify each photo as a type of fruit. "type of fruit" is the target feature (aka "number of targets: 1"), and "more than 2 possible outcomes" (aka "target cardinality: >2") because it might be an apple, or a pear, or an orange (or others 1)

weight ... ... type of fruit
. . . apple
. . . pear
. . . apple
. . . orange

The target feature might be "one-hot encoded", (i.e. if you use a scikit-learn LabelBinarizer), so it looks like the table below, which is identical to the data above. Even though there are three (or more) output columns, there is still only "one target" (type of fruit).

weight ... ... fruit:apple fruit:pear fruit:orange
. . . 1 0 0
. . . 0 1 0
. . . 1 0 0
. . . 0 0 1

If your data is one-hot encoded like the above, you know it's a multiclass problem when the output can have a "1" in only one of the output columns; in other words, the classes are "mutually exclusive". In this case, you can only be one type of fruit 2. As the scikit-learn says:

Multiclass classification assumes that each sample is assigned to one and only one label - one sample cannot, for example, be both a pear and an apple.

In other words, the data below is nonsense -- is it an apple or a pear?

weight ... ... fruit:apple fruit:pear fruit:orange
. . . 1 1 0

Multilabel classification

Multiple target features, each with a binary outcome

For example, imagine you build a predictor to analyze documents and tag each document with various topics (aka "number of targets: >1"). The multiple target features mean one for each topic like "religion", "politics", "finance", etc. Each has a binary outcome because you're either tagged with that topic or not (aka "target cardinality: 2 (0 or 1)" 3).

term frequency ... ... topic:religion topic:finance topic:education
. . . 0 1 1
. . . 1 0 1
. . . 0 0 0
. . . 1 1 1

Notice the difference from the multiclass example.

  • In multiclass "one-hot encoded" outputs, the output columns are mutually exclusive (only one column can have a "1" in it)
  • In multilabel, a single document might be tagged in with multiple topics aka target features (multiple columns can have a "1" in it)
    • For example, the first row was tagged with "finance" and "education"; maybe the row represents an article on teaching students (education) how to make a budget (finance)

Notice another difference from multiclass:

  • In multiclass, exactly one column has a "1" in it
  • In multilabel, it's possible that no columns have a "1"; all columns are "0"
    • For example, the third row was not tagged with anything; maybe the document is a poem, and it is hard to tag with any of the topics

Multiclass-multioutput classification

Multiple target features, each with more than 2 possible outcomes

This is also known as "multitask classification". Consider the example of the fruit again (just a "multiclass" problem).

  • Instead of a single target feature (aka "number of targets: 1"), like trying to predict what type of fruit...
  • now we have multiple target features ("number of targets: > 1")...

For example, let's predict the color in addition to the type of fruit. In this way, it becomes like a multilabel problem.

However,

  • unlike a multilabel problem where we only have binary outcomes (aka "target cardinality: 2 (0 or 1)") -- like whether a topic applies or not...
  • in this case we can predict multiple outcomes, like "color:green", "color:orange", "color:red" in addition to "fruit:apple", "fruit:pear", "fruit:orange", etc...
weight ... ... fruit type color
. . . apple green
. . . orange orange
. . . pear green

This answer calls it a "multioutput-multiclass" problem.

What about "one-hot-encoding" the target columns like we did in the multiclass example? The scikit-learn multitask "Target format" section says the target format is:

a dense matrix of shape (n_samples, n_classes)

... as shown in our example above

  • n_samples = 3 rows
  • n_classes = 2 columns ( fruit type, color)

... but the scikit-learn "glossary" suggests that for multioutput classification, you could use

a list of 2d arrays corresponding to each multiclass decision function

That is, you could use one-hot encoding...

Multioutput Regression

Multiple target features, each with a continuous set of possible outcomes

With multiclass-multioutput (aka "multitask"), you are dealing with a classification problem; you are predicting a set of discrete outcomes (aka "target cardinality: >1"). With multioutput regression, you are dealing with a regression problem and a set of continuous outcomes (aka "target cardinality: Continuous").

For example, instead of predicting what type of fruit or what color (discrete outcomes), imagine you were predicting how many calories were in the fruit or the shelf life (how long the fruit lasted before going bad):

weight ... ... calories shelf life (days)
. . . 220 23
. . . 300 40
. . . 180 20

Notice the target features are now continuous values.

General SciKit notes

Note:

All classifiers in the scikit-learn do multiclass classification out-of-the-box. You don’t need to use the sklearn.multiclass module unless you want to experiment with different multiclass strategies.

Turning a "binary classifier" into a "multiclass classifier" is possible using "One-vs-One" or "One-vs-Rest" strategies, discussed on the scikit-learn and elsewhere.

Note:

... all classifiers handling multiclass-multioutput (also known as multitask classification) tasks support the multilabel classification task as a special case.

I think this is what the accepted-answer author is saying, that SciKit Learn's description of "multilabel" meaning only "target cardinality:2 (0 or 1)" aka binary -- is just a specialization of "multiclass-multioutput"; (what the accepted-answer author would call "multilabel"!)

Note the warning:

At present, no metric in sklearn.metrics supports the multiclass-multioutput classification task.

This means the scikit-learn library doesn't have a way to evaluate how well your "multiclass-multioutput" predictor is doing. How can you evaluate? Maybe a series of classification evaluations?

Note SciKit Learn says:

[A multilabel approach] is comparable to running n_classes binary classification tasks, ... which treats each label independently, whereas multilabel classifiers may treat the multiple classes simultaneously, accounting for correlated behavior among them.

This is echoed in the other answer here, where "multi-label requires the model to learn correlation"

1 What if the multiclass predictor is given a banana photo? Not a pear, not an apple, not an orange? As the other answer says, the classes "taken together should be comprehensive"

Well ...

  • you could define the universe for your problem as existing only of apples, pears, and oranges -- so your predictor would pick whichever of those was closest to a banana...
  • or you could add "banana" to the domain of your fruits and train a new model that can recognize bananas...
  • or you could add an "other" class, like: "not an apple, nor pear, nor orange", and a banana could fall into that category (alongside a pineapple, a kumquat, etc!)

2 You might ask, what if the multiclass predictor is 80% sure it's an apple, but gives a 15% chance it's a pear, and a 5% chance it's an orange? You might be referring to "soft" scores (vs "hard" scores), aka the "probabilities" of each class (predict_proba() vs predict()). Probabilities are possible in the scikit-learn depending on the classifier you're dealing with (see here for RandomTree classifiers), but we don't discuss it here; we're concerned with hard predictions.

3 As part of a multilabel problem, the accepted answer gives the crab data, and the two target features "sex" (M or F) and "species" (B or O). I was tempted to say, "species/color is not binary! There are more than two colors, hence this is not a multilabel problem", but in the case of the crab dataset, I guess species is binary: there are only two possible colors: Blue or Orange. The accepted answer author refutes the "binary" requirement for the "Multilabel" problem. I believe the author is extending it into a "multiclass-multioutput classification" problem.

$\endgroup$
1
$\begingroup$

Multi Class classification Problem One right answer and Mutually exclusive outputs(eg iris, numbers) Multi Label Classification more than one right answer and appropriate output or Non exclusive eg(sugar test, eye test)

In multi class we user softmax In multi label we use sigmoid

$\endgroup$
1
  • 1
    $\begingroup$ The last sentence isn't generally the case, e.g. one of the labels could also be multiclass or continuous for regression. $\endgroup$
    – doubllle
    Oct 7, 2020 at 7:20

Not the answer you're looking for? Browse other questions tagged or ask your own question.