How to intuitively understand difference between algorithmic adaptation vs problem transformation in multi-label classification?

Question

I am trying to understand how multi-label classification is done and rather got confused with what wikipedia has to say, I am directly quoting it:

"Several problem transformation methods exist for multi-label classification; the baseline approach, called the binary relevance method,[1][2][3] amounts to independently training one binary classifier for each label. Given an unseen sample, the combined model then predicts all labels for this sample for which the respective classifiers predict a positive result. This method of dividing the task into multiple binary tasks has something in common with the one-vs.-all (OvA, or one-vs.-rest, OvR) method for multiclass classification. Note though that it is not the same method: in binary relevance we train one classifier for each label, not one classifier for each possible value for the label.

Various other transformations exist. Of these, the label powerset (LP) transformation creates one binary classifier for every label combination attested in the training set.[1] The random k-labelsets (RAKEL) algorithm uses multiple LP classifiers, each trained on a random subset of the actual labels; prediction using this ensemble method proceeds by a voting scheme.[4]"

Can someone provide a intuitive example, given a some samples with multi-labels how the classification is done?

More precisely how is algorithmic adaptation method is different from problem transformation? Please compare Binary Relevance (as a representative of problem transformation) with Multi class SVM/any classifier trained on single label (as a representative of algorithmic adaptation)

---

Given some training samples having multi-labels, how can that be transformed to single labels? For example, according to my understanding, if you know that the labels for each of the training sample is arranged in a descending order of their occurrence probability then you can just consider the first label as the most possible single label for that sample. The second way is to make copies of that sample for each corresponding labels. Third way is to consider each set of labels as a single class. Which one is a preferred way?

For algorithmic adaptation method, I guess you don't need n-single class classifiers just the way you need it for problem transformation method. So, its a single classifier capable of providing multiple outputs.

Am I making sense?

Answer 1

Let's see.

Multi-label classification is the problem of assigning multiple labels (categories) to each input sample.

The classical example is blog posts and tags. Say you want to train an algorithm to try and figure out what tags a StackOverflow (SO) question will be assigned by its author, given only its title and body.

You algorithm needs to provide multiple outputs for each input sample. So, for instance, you want your algorithm to be able to assign all tags it thinks that SO question would have in real life.

Can you think of a dead simple way to do that using regular machine learning tools?

You probably have come up with what's called Binary Relevance. It's an obvious way to transform the problem so that you can use regular ML tools.

You just train a binary classifier for each tag individually and, for each classifier that returns True, you assign that tag to that input sample. This is the binary relevance method (note that this is also called OvR (one versus rest) in toolkits such as scikit-learn).

Binary relevance is a problem transformation method because it's equivalent to transforming a single input sample with 4 tags into 4 separate input samples, one for each tag. After transforming the problem like this, you can use any single-label machine learning algorithm.

This is different from algorithm adaptation methods, where you don't just transform the dataset and then use any old single-label ML algorithm. These are algorithms specifically tailored for multi-label classification; these are multi-label adaptations for many popular algorithms such as neural nets, kNN, random forests, etc.

All Stackexchange forums (like this one) have public datasets which are good ways to train multi-label algorithms.