Questions tagged [multi-class]

Multiclass classification is a classification task in which there are more than two classes. It is also called multinomial classification.

Filter by
Sorted by
Tagged with
81
votes
8answers
103k views

How to compute precision/recall for multiclass-multilabel classification?

I'm wondering how to calculate precision and recall measures for multiclass multilabel classification, i.e. classification where there are more than two labels, and where each instance can have ...
100
votes
3answers
155k views

How do you calculate precision and recall for multiclass classification using confusion matrix?

I wonder how to compute precision and recall using a confusion matrix for a multi-class classification problem. Specifically, an observation can only be assigned to its most probable class / label. I ...
71
votes
6answers
56k views

What is the difference between Multiclass and Multilabel Problem

What is the difference between a multiclass problem and a multilabel problem?
18
votes
2answers
17k views

Best way to perform multiclass SVM

I know that the SVM is a binary classifier. I would like to extend it to multi-class SVM. Which is the best, and maybe the easiest, way to perform it? code: in MATLAB ...
6
votes
1answer
7k views

Micro vs weighted F1 score

In a multi-label or multi-class classification setting, when choosing between a micro or a weighted F1 score, what shall I take into account? The main upside of choosing macro is that one gets a ...
23
votes
2answers
8k views

How to handle the difference between the distribution of the test set and the training set?

I think one basic assumption of machine learning or parameter estimation is that the unseen data come from the same distribution as the training set. However, in some practical cases, the distribution ...
6
votes
2answers
20k views

Multi-class logarithmic loss function per class

In a multi-classification problem, we define the logarithmic loss function $F$ in terms of the logarithmic loss function per label $F_i$ as: $$ F = -\frac{1}{N}\sum_{i}^{N}\sum_{j}^{M}y_{ij} \cdot Ln(...
6
votes
2answers
4k views

How to threshold multiclass probability prediction to get confusion matrix?

Lets say my multinomial logistic regression predict that a chance of a sample belonging to a each class is A=0.6, B=0.3, C=0.1 How do I threshold this values to get just binary prediction of a sample ...
3
votes
1answer
202 views

Training N classifiers for N labels vs one classifier with N labels

I have a classification problem which is multi-label with N labels. I would like to know which method would be the better choice? Training N classifiers (1 for each label) or a single classifier which ...
1
vote
1answer
941 views

How to build ROC curve (or AUC) of classification model from confusion matrix only

I've been looking into using ROC curves as a evaluation tool of a multi-class classification. The only data I have about this classification is in form of 7-by-7 confusion matrix. Visualisation of ...
35
votes
3answers
39k views

How to determine the quality of a multiclass classifier

Given a dataset with instances $x_i$ together with $N$ classes where every instance $x_i$ belongs exactly to one class $y_i$ a multiclass classifier After the training and testing I basically have a ...
11
votes
1answer
10k views

Extending 2-class models to multi-class problems

This paper on Adaboost gives some suggestions and code (page 17) for extending 2-class models to K-class problems. I would like to generalize this code, such that I can easily plug in different 2-...
10
votes
1answer
4k views

How the probability threshold of a classifier can be adjusted in case of multiple classes? [duplicate]

The above is a very simple example of having a probability classifier output for a binary-class case either 0 or 1 based on some probabilities. In addition it is straightforward how you can change the ...
11
votes
2answers
19k views

Output of Scikit SVM in multiclass classification always gives same label

I am currently using Scikit learn with the following code: ...
6
votes
1answer
4k views

Multi-label or multi-class…or both?

I'm having a hard time getting the difference between multi-class and multi-label classification with CNNs. My understanding is that if I want to classify different breeds of dogs, that is a multi-...
5
votes
0answers
1k views

How to include negative examples in multi-class classification?

I have a problem similar to this question: How do I use negative examples (in addition to positive ones) for training a multiclass softmax classifier (or a neural net with softmax output)? where I ...
2
votes
1answer
2k views

Probability calibration metric for multiclass classifier

A machine learning classifier can be calibrated so that when the probability that datapoint i is of class A is 0.6, this is true 60% of the time. In the binary class setting, this can be visualised ...
6
votes
1answer
4k views

Softmax regression or $K$ binary logistic regression

For a multi-class classification problem, we can use $K$ binary logistic classifiers, or one softmax regression classifier, so how to make the choice between the two? IMHO, the $K$ binary logistic ...
5
votes
2answers
2k views

Why do one-versus-all multi class SVMs need to be calibrated?

On the wiki page for multi-class support vector machines (https://en.wikipedia.org/wiki/Support_vector_machine#Multiclass_SVM) it states that "it is important that the output functions be calibrated ...
2
votes
1answer
2k views

How do I use negative examples (in addition to positive ones) for training a multiclass softmax classifier (or a neural net with softmax output)?

Suppose we are training a neural network for multi-class classification, and we use softmax (or hierarchical softmax) as its output layer. For positive examples, we need to maximize the log ...
1
vote
0answers
384 views

Evaluating Unbalanced Multiclass Classifiers: Which Tests to Use? [closed]

I am looking for some comprehensive instructions and ideally out of the box solutions (ideally for python) for evaluating different classifiers (which are already trained) for a multiclass ...
6
votes
3answers
5k views

How can I derive confidence intervals from the confusion matrix for a classifier?

I have am using k-fold cross validation to generate a confusion matrix for a classifier. I need to calculate 95% confidence intervals for the number of times each class is predicted when run against a ...
4
votes
0answers
961 views

Calibrating multiple binary SVM classifiers for one-vs-all multi-class classification

I'm classifying text using the one-vs-all approach. There are three classes. I've trained 3 different binary SVM classifiers using 10-fold cross-validation. The accuracy of the binary classifiers ...
2
votes
1answer
4k views

Calculate accuracy using true/false positives/negatives

I got predicted = [0, 0, 1, 0, 1, 2, 1, 0, 1, 0] actual = [1, 2, 1, 2, 0, 1, 0, 2, 1, 1] from multiclass classifier Next, I calculate for 3 classes ...
2
votes
0answers
161 views

Streaming multi-class classification with growing number of classes

I'm looking for some references for online multi-class classification problem where the number of classes grows over time. Concretely, the data at time-step $t$ comes in the form $\left(\mathbf{x}_t, ...
1
vote
3answers
559 views

Which is the best classifier and with what performance measures?

I tried to implement a Classifier comparison like in the scikit-learn for text classification. I used an 81 instances as a training sample and a 46 instances as a test sample. I tried several ...
1
vote
0answers
777 views

How to combine/aggregate classification accuracies from binary one-vs-one classifiers to get final accuracy equivalent a multiclass classifier?

Consider a 3 class data, say, Iris data. Suppose we want do binary SVM classification for this multiclass data using Python's sklearn. So we have the following ...
0
votes
1answer
94 views

Multi-class evaluation: found different macro F1 scores, which one to use?

I want to evaluate my multi-class classifier against a gold reference and obtain a single score that reflects its performance. In my data I have many classes that are important but rare, so I was ...