0
votes
0answers
10 views

Multi-class logarithmic loss function per class

In a multi-classification problem, we define the logarithmic loss function $F$ in terms of the logarithmic loss function per label $F_i$ as: $$ F = -\frac{1}{N}\sum_{i}^{N}\sum_{j}^{M}y_{ij} \cdot ...
0
votes
0answers
2 views

How can I counteract the effect of a degenerate classifier in an OVA Model?

Suppose I build a OVA classification model for classification with more than 2 possible classes (a model of sub-models, where each submodel predicts the probability of a data point belonging in a ...
0
votes
0answers
13 views

Implementation of batch Hieron

I would like to implement the Hieron algorithm, which is described in the paper "Large Margin Hierarchical Classification". The basic online Hieron is specified in the paper, but I need the batch ...
1
vote
0answers
24 views

Walking recognition

I have walking samples from 20 different people. My aim is to detect which walking samples are from which person. I'm trying to achieve this by extracting "walking cycles" from each person's dataset ...
0
votes
0answers
24 views

Softmax regression or $K$ binary logistic regression

For a multi-class classification problem, we can use $K$ binary logistic classifiers, or one softmax regression classifier, so how to make the choice between the two? IMHO, the $K$ binary logistic ...
2
votes
1answer
91 views

What is the difference between a multi-label and a multi-class classification?

What is the difference between multi-label classification and multiclass classfication. Speficially, what is the difference between a label and a class? Please provide a clear example. "Multiclass ...
1
vote
0answers
68 views

What should I use - Multi label classification or Multi class classification?

In my dataset, I have 2 labels, positive and negative. Most samples belong to only one class, either positive or negative. A small fraction of samples take both labels i.e. both positive and negative. ...
0
votes
0answers
9 views

Precision of an incomplete classifier

Given a testing set of nodes which can be either +,-, or 0, I use an incomplete classifier which allows me to predict if a node is +, -, 0, not +, not -, or not 0, and sometimes it cannot predict ...
1
vote
2answers
51 views

Can a nuisance multi-class classifier do better than binary classifier?

This is rather a theoretical question in order to save the trouble in trying to do empirical testing and is part of a bet, so I hope I am right... Say there are M classes in the data BUT you want to ...
0
votes
0answers
20 views

How to extract the weight and bias in libsvm?

I want to extract the weight and the bias for my dataset which has two columns and more than 3000 rows. Also I want to draw a line by this equation: y = w.x + b where w is the weight, x is the data ...
1
vote
1answer
72 views

What is the difference between accuracy and agreement?

According to Manning et al. (p. 155) accuracy is the sum of the diagonal in the confusion matrix divided by the sum of all items. On the other hand, following Artstein and Poesio (p . 558) precisely ...
0
votes
0answers
29 views

Implementing MMAC algorithm

I was reading a paper by Fadi A. Thabtah and et al., "MMAC: A New Multi-class, Multi-label Associative Classification Approach". I was trying to implement their MMAC algorithm. In MMAC algorithm that ...
0
votes
0answers
56 views

Significance test for multiclass classifier

In a multiclass classification problem, I want to measure the significance of my classifier against the null hypothesis (in this case, chance level). In this paper, in section 3.4, for a binary ...
0
votes
0answers
42 views

Learning from distance matrices

I have a task to build multi-task classification model based on distance (similarity) matrix only. They are already precalculated and no changes here can be applied. Can you, please, recommend me ...
2
votes
1answer
68 views

libsvm_linear kernel_increasing C value

I'm using libsvm in C-SVC mode (-s= 0) with linear kernel (-t= 0), and I'm required to train multiple SVMs( I have four classes). My training and test sets have the same number of instances and ...
2
votes
1answer
137 views

Using predict_proba with sklearn's multiclass SVC

I'm using python's sklearn for multi-class classification (SVC) When using the predict method, i get very high scores with my dataset, However, I want to plot ROC curves for each of my classes. That ...
1
vote
3answers
59 views

“finely” labeled classification problem

I came across a kind of classification problem. Suppose I have a dataset with labels, where each label is one of a1, a2, a3, b1, and b2. I want to make a classifier that estimates "coarse" label, a ...
3
votes
2answers
115 views

Good strategy for multiclass classification (when there is hierarchical class structure )

Combining binary classifiers, I want to solve multi-class classification problem in the following setting. Suppose there is a dataset and each data is in one of four classes: A1, A2, B1 and B2. A1 and ...
0
votes
0answers
70 views

Accuracy vs F1 Measure in Multilabel Classification

I'd like to evaluate a multilabel classification algorithms and I was thinking of using both Accuracy and F1-Measure with: ...
2
votes
1answer
129 views

Area Under ROC Curve for Multiple Classes

I am working with a highly class-skewed three class classification problem. The class percentages are A = 1.8%, B = 17.5% and C = 80.7%. According to this paper, the following definition of ...
1
vote
1answer
129 views

Unassigned classes in Multiclass SVM with One vs All Approach

How do you handle unassigned classes in multiclass support vector machines (multiclass SVM) with the One vs All approach? Lets say my training data has three classes A, B, and C. I use 3 SVM ...
0
votes
1answer
62 views

Transform multiclass classification to binary - benefits?

I have 400 instances which must be categorized into 4 classes. Using WEKA, I tried out a couple of multiclass classifiers like J48 and Random Forests, but never made it above Kappa 0.6 and ~65% ...
3
votes
2answers
147 views

Machine learning for multi-level response

I have a dataset with ~90000 observations and less than 10 features (all continuous). The problem is that the response variable has ~300 categories. Currently I would try to fit a multinomial linear ...
0
votes
1answer
184 views

How to improve classification performance based on multiple known classification results

I am working on a classification problem, which may contains a unknown number of data classes, typically 5-50 classes in each sample. I had several classification algorithms, each gives me a ...
2
votes
1answer
154 views

Hierarchical classification where leaf nodes in a tree are at no particular level

I have a set of hierarchical classes (ex. "object/architecture/building/residential building/house/farmhouse"), and I build a tree where each node is a classifier. However, the appropriate class for a ...
5
votes
2answers
300 views

How can a multiclass perceptron work?

I don't have any background in math, but I understand how the simple Perceptron works and I think I grasp the concept of a hyperplane (I imagine it geometrically as a plane in 3D space which seperates ...
3
votes
2answers
265 views

Named entity recognition and class imbalance

I have implemented Maximum-entropy Markov model (MEMM) for the Named entity recognition (NER) problem. I have four classes: geographical, people, material (book titles etc) and other. Class ...
4
votes
1answer
340 views

Confusion matrices with percentages rather than number of instances?

Most of the confusion matrices I've seen contain the number of instances in each cell. Isn't a confusion matrix with the percentage of instances in each cell easier to read? Is this approach wrong or ...
3
votes
0answers
117 views

binomial test for testing significance of classification very sensitive to hits count

I'm doing a multi-class classification task and I wonder if I'm doing the binomial test correctly since it is very sensitive to the count of correctly classified trials (hits count). Say, there are ...
17
votes
3answers
10k views

How to calculate precision and recall for multiclass classification using confusion matrix?

I wonder how to compute precision and recall using a confusion matrix for a multi-class classification problem. In specific, one observation can only be assigned with most probable class / label. I ...
3
votes
0answers
161 views

Reasons for transforming multiple class classification problem into a set of binary sub-problems?

Does anyone know of a good reference that list the reasons for transforming multiple class classification problem into a set of binary sub-problems? In response to comment: One reason to transform a ...
10
votes
3answers
1k views

How to determine the quality of a multiclass classifier

Given a dataset with instances $x_i$ together with $N$ classes where every instance $x_i$ belongs exactly to one class $y_i$ a multiclass classifier After the training and testing I basically have ...
10
votes
1answer
417 views

How to handle the difference between the distribution of the test set and the training set?

I think one basic assumption of machine learning or parameter estimation is that the unseen data come from the same distribution as the training set. However, in some practical cases, the distribution ...
3
votes
0answers
106 views

Hierarchical decomposition of an imbalanced multiclass classification problem

I have a heavily imbalanced multiclass text classification problem: one class is very probable a priori (P), while the remaining four ones are about equally ...
0
votes
1answer
148 views

How do I do multiscale HMM classification?

I'm using hidden Markov models to classify some accelerometer data. I take the Fourier transform of the raw data at a given window length, and then train an HMM for each class, and every test instance ...
5
votes
3answers
648 views

Suitable number of classes for SVM in text categorization

I'm doing text categorization with R and SVM in the package e1071. I have around 30000 text files for training, and 10000 for test. The goal is to hierarchically categorize these files. For example, I ...
1
vote
0answers
158 views

Binary Classification of Multiple Groups

I've ran across a type of classification problem that I don't think fits into the traditional multi-class framework. Just wanted to run it across you guys to see if you had any ideas. So Lets say we ...
4
votes
0answers
155 views

Number of states and symbols in multi class Hidden Markov Model classifier

I'm designing a multi class classifier (for 4 classes) using Discrete HMMs with States N and Symbols M for each of the HMM. However, I found that recognition performance(i.e highest log likelihood) ...
4
votes
2answers
340 views

Predicting multiple targets or classes?

Suppose I am building a predictive model where I am trying to predict multiple events (for instance, both the roll of a die and the toss of a coin). Most algorithms that I am familiar with work with ...
27
votes
3answers
8k views

How to compute precision/recall for multiclass-multilabel classification?

I'm wondering how to calculate precision and recall measures for multiclass multilabel classification, i.e. classification where there are more than two labels, and where each instance can have ...
0
votes
2answers
668 views

Voting in Multi-Class SVM [duplicate]

I am currently trying to classify 6 class of facial expression using SVM. I am using MATLAB and LIBSVM to do my classification. The problem i face is the pred label below produces 0 and 1. It treats ...
9
votes
1answer
2k views

Extending 2-class models to multi-class problems

This paper on Adaboost gives some suggestions and code (page 17) for extending 2-class models to K-class problems. I would like to generalize this code, such that I can easily plug in different ...