74 votes

What is the difference between Multiclass and Multilabel Problem

Multiclass classification means a classification task with more than two classes; e.g., classify a set of images of fruits which may be oranges, apples, or pears. Multiclass classification makes the ...
  • 1,101
31 votes

Multilabel classification metrics on scikit

The subset accuracy is indeed a harsh metric. To get a sense of how good or bad 0.29 is, some idea: look at how many labels you have an average for each sample look at the inter-annotator agreement, ...
28 votes

What is the difference between Multiclass and Multilabel Problem

To complement the other answers, here are some figures. One row = the expected output for one sample. Multiclass One column = one class (one-hot encoding) Multilabel One column = one class You ...
27 votes

How to compute precision/recall for multiclass-multilabel classification?

For multi-label classification you have two ways to go First consider the following. $n$ is the number of examples. $Y_i$ is the ground truth label assignment of the $i^{th}$ example.. $x_i$ is the $...
  • 431
20 votes

How do you calculate precision and recall for multiclass classification using confusion matrix?

Using sklearn or tensorflow and numpy: ...
18 votes

How to compute precision/recall for multiclass-multilabel classification?

Here is some discuss of coursera forum thread about confusion matrix and multi-class precision/recall measurement. The basic idea is to compute all precision and recall of all the classes, then ...
  • 181
16 votes

Vowpal Wabbit: best strategy for short text data like titles & kewords

Here are some tips for enhancing the performance of VW models: Shuffle the data prior to training. Having a non-random ordering of your dataset can really mess VW up. You're already using multiple ...
  • 22.6k
15 votes

How to build a confusion matrix for a multiclass classifier?

While there are some answers already on this forum I thought I'd give the explicit equations to make it more definite: Assuming you have a multi-class confusion matrix of the form, \begin{align} C=\...
14 votes
Accepted

Many binary classifiers vs. single multiclass classifier

Your Option 1 may not be the best way to go; if you want to have multiple binary classifiers try a strategy called One-vs-All. In One-vs-All you essentially have an expert binary classifier that is ...
14 votes
Accepted

Multi-label or multi-class...or both?

Definitions. In a classification task, your goal is to learn a mapping $h: X\rightarrow Y$ (with your favourite ML algorithm, e.g CNNs). We make two common distinctions: Binary vs multiclass: In ...
  • 2,242
13 votes
Accepted

Output of Scikit SVM in multiclass classification always gives same label

A likely cause is the fact you are not tuning your model. You need to find good values for $C$ and $\gamma$. In your case, the defaults turn out to be bad, which leads to trivial models that always ...
  • 17.7k
12 votes
Accepted

How the probability threshold of a classifier can be adjusted in case of multiple classes?

You can use a prior distribution over the classes. Let us assume that your model computes a vector of class probabilities $v$. You can define a vector of prior probabilities $\pi$ and then compute ...
  • 2,141
11 votes

Output of Scikit SVM in multiclass classification always gives same label

The problem does turn out to be parameter testing. I did not try when gamma is between 0.0 (which is 1/n_feature) and 1. On my data ...
11 votes
Accepted

Is GINI limited to binary classifiers or can we use it for multi-class classifiers as well?

The Gini impurity can definitely be used to quantify variance in a multi-class setting, not only in the binary case. Gini impurity is defined as $$ G(p) = \sum_{i=1}^{J}{p_i} \sum_{k \neq i}^{J}{p_k}...
  • 278
9 votes
Accepted

How to build ROC curve (or AUC) of classification model from confusion matrix only

You cannot construct a ROC curve from the confusion matrix alone. A confusion matrix represents a single point in the ROC space, and you need all possible confusion matrices at all thresholds to build ...
  • 3,089
9 votes
Accepted

Poor multiclass classification using Caret in R

I think that the problem is not that you are using the classification methods poorly, but rather that this data has little predictive power for the regions. First of all, two classes have little ...
  • 2,513
8 votes
Accepted

Why do one-versus-all multi class SVMs need to be calibrated?

Setup Recall that an SVM can be viewed as a weight vector $w$ and an intercept $b$, and that the output function for a test input $x$ is is $\langle w, x \rangle + b$. To get a binary prediction, we ...
  • 22.6k
8 votes

What is the difference between Multitask and Multiclass learning

Just to give a more clear understanding, I have explained each terminology with examples Multiclass classification/(One-Vs-One and One-Vs-ALL): Classification task with >2 classes! ...
  • 653
8 votes
Accepted

How to apply Softmax as Activation function in multi-layer Perceptron in scikit-learn?

I suposse that the Softmax function is applied when you request a probability prediction by calling the method mlp.predict_proba(X). To support my supposition I ...
8 votes

Neural network for multi label classification with large number of classes outputs only zero

Tensorflow has a loss function weighted_cross_entropy_with_logits, which can be used to give more weight to the 1's. So it should be applicable to a sparse multi-...
  • 263
7 votes

Matthews correlation coefficient with multi-class

Yes, in general, you can. This approach you want to use is sometimes called "Micro-Averaging": first, sum all TNs, FPs, etc for ...
7 votes

How to threshold multiclass probability prediction to get confusion matrix?

According to @cangrejo's answer: https://stats.stackexchange.com/a/310956/194535, suppose the original output probability of your model is the vector $v$, and then you can define the prior ...
7 votes

How to apply Softmax as Activation function in multi-layer Perceptron in scikit-learn?

The MLPClassifier can be used for "multiclass classification", "binary classification" and "multilabel classification". So the output layer is decided based on type of Y : Multiclass: The outmost ...
7 votes

Better performance using Random Forest one-Vs-All than Random Forest multiclass?

I had exactly the same question as you, and was a bit sad to find out no answers were posted on your topic... That said, I found this paper : One-Vs-All Binarization Technique in the Context of ...
  • 171
7 votes
Accepted

Error while performing multiclass classification using Gridsearch CV

Accuracy might look tempting but not a good metric in general. In multilabel classification, for each class we'll have f1 score, ...
  • 53.8k
6 votes

How to compute accuracy for multi class classification problem and how is accuracy equal to weighted precision?

I've got a wonderful solution and a perfect understandable solution for this problem as I was looking for same from this Question You can calculate and store accuracy with: ...
  • 171
6 votes
Accepted

Imbalanced multiclass classification with many classes

There is no real answer to your question, because it really depends on what you are trying to archive, i.e. is your goal to get a very high classification accuracy or is it rather data exploration? ...
  • 1,015
6 votes
Accepted

Optimize classification rule in multinomial logistic regression

The question is ill-posed. If you are trying to optimise some function of specificity and sensitivity (other than accuracy) for a logistic regression model by altering the classification threshold, ...
5 votes
Accepted

Suggestions needed about classifier fusion

There are several approaches to combine classifiers, and in your case, what you are doing is one of them. Also, it is reasonable to expect an improvement in the combined classifier compared to either ...
5 votes

Maximum number of classes for RandomForest multiclass estimation

I have at least one experience doing so. For the NHTS 2017 dataset, I have modeled a number of variables. Notably, random forests perform quite well on predicting vehicle ownership per household (...
  • 2,127

Only top scored, non community-wiki answers of a minimum length are eligible