We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [classification]

Statistical classification is the problem of identifying the sub-population to which new observations belong, where the identity of the sub-population is unknown, on the basis of a training set of data containing observations whose sub-population is known. Therefore these classifications will show a variable behavior which can be studied by statistics.

Filter by
Sorted by
Tagged with
112
votes
7answers
38k views

Why is accuracy not the best measure for assessing classification models?

This is a general question that was asked indirectly multiple times in here, but it lacks a single authoritative answer. It would be great to have a detailed answer to this for the reference. ...
53
votes
5answers
10k views

When is unbalanced data really a problem in Machine Learning?

We already had multiple questions about unbalanced data when using logistic regression, SVM, decision trees, bagging and a number of other similar questions, what makes it a very popular topic! ...
75
votes
3answers
18k views

Why isn't Logistic Regression called Logistic Classification?

Since Logistic Regression is a statistical classification model dealing with categorical dependent variables, why isn't it called Logistic Classification? Shouldn't the "Regression" name be reserved ...
49
votes
4answers
31k views

Classification probability threshold

I have a question regarding classification in general. Let f be a classifier, which outputs a set of probabilities given some data D. Normally, one would say: well, if P(c|D) > 0.5, we will assign a ...
136
votes
4answers
92k views

Choice of K in K-fold cross-validation

I've been using the $K$-fold cross-validation a few times now to evaluate performance of some learning algorithms, but I've always been puzzled as to how I should choose the value of $K$. I've often ...
80
votes
5answers
114k views

How to calculate Area Under the Curve (AUC), or the c-statistic, by hand

I am interested in calculating area under the curve (AUC), or the c-statistic, by hand for a binary logistic regression model. For example, in the validation dataset, I have the true value for the ...
232
votes
5answers
346k views

What does AUC stand for and what is it?

Searched high and low and have not been able to find out what AUC, as in related to prediction, stands for or means.
25
votes
5answers
4k views

How can top principal components retain the predictive power on a dependent variable (or even lead to better predictions)?

Suppose I am running a regression $Y \sim X$. Why by selecting top $k$ principle components of $X$, does the model retain its predictive power on $Y$? I understand that from dimensionality-reduction/...
35
votes
3answers
20k views

PCA and the train/test split

I have a dataset for which I have multiple sets of binary labels. For each set of labels, I train a classifier, evaluating it by cross-validation. I want to reduce dimensionality using principal ...
66
votes
4answers
88k views

How to plot ROC curves in multiclass classification?

In other words, instead of having a two class problem I am dealing with 4 classes and still would like to assess performance using AUC.
30
votes
1answer
5k views

Do we have to tune the number of trees in a random forest?

Software implementations of random forest classifiers have a number of parameters to allow users to fine-tune the algorithm's behavior, including the number of trees in the forest. Is this a parameter ...
76
votes
6answers
30k views

Feature selection for “final” model when performing cross-validation in machine learning

I am getting a bit confused about feature selection and machine learning and I was wondering if you could help me out. I have a microarray dataset that is classified into two groups and has 1000s of ...
12
votes
2answers
13k views

Linear discriminant analysis and Bayes rule: classification

What is the relation between Linear discriminant analysis and Bayes rule? I understand that LDA is used in classification by trying to minimize the ratio of within group variance and between group ...
109
votes
5answers
57k views

How does a Support Vector Machine (SVM) work?

How does a Support Vector Machine (SVM) work, and what differentiates it from other linear classifiers, such as the Linear Perceptron, Linear Discriminant Analysis, or Logistic Regression? * (* I'm ...
43
votes
4answers
36k views

Training a decision tree against unbalanced data

I'm new to data mining and I'm trying to train a decision tree against a data set which is highly unbalanced. However, I'm having problems with poor predictive accuracy. The data consists of students ...
29
votes
3answers
26k views

Why is AUC higher for a classifier that is less accurate than for one that is more accurate?

I have two classifiers A: naive Bayesian network B: tree (singly-connected) Bayesian network In terms of accuracy and other measures, A performs comparatively worse than B. However, when I use the R ...
19
votes
1answer
11k views

How LDA, a classification technique, also serves as dimensionality reduction technique like PCA

In this article , the author links linear discriminant analysis (LDA) to principal component analysis (PCA). With my limited knowledge, I am not able to follow how LDA can be somewhat similar to PCA. ...
7
votes
2answers
3k views

How to use boxplots to find the point where values are more likely to come from different conditions?

I have plotted some data using box plots. I am comparing Condition 1 (left) and Condition 2 (Right) values. My aim is to find a point at which we make a decision where the value changes from point ...
77
votes
1answer
7k views

Help me understand Support Vector Machines

I understand the basics of what a Support Vector Machines' aim is in terms of classifying an input set into several different classes, but what I don't understand is some of the nitty-gritty details. ...
52
votes
6answers
25k views

Binary classification with strongly unbalanced classes

I have a data set in the form of (features, binary output 0 or 1), but 1 happens pretty rarely, so just by always predicting 0, I get accuracy between 70% and 90% (depending on the particular data I ...
24
votes
3answers
16k views

Cross-validation or bootstrapping to evaluate classification performance?

What is the most appropriate sampling method to evaluate the performance of a classifier on a particular data set and compare it with other classifiers? Cross-validation seems to be standard practice, ...
25
votes
2answers
16k views

Does it make sense to combine PCA and LDA?

Assume I have a dataset for a supervised statistical classification task, e.g., via a Bayes' classifier. This dataset consists of 20 features and I want to boil it down to 2 features via ...
20
votes
2answers
3k views

What problem does oversampling, undersampling, and SMOTE solve?

In a recent, well recieved, question, Tim asks when is unbalanced data really a problem in Machine Learning? The premise of the question is that there is a lot of machine learning literature ...
133
votes
4answers
117k views

Cohen's kappa in plain English

I am reading a data mining book and it mentioned the Kappa statistic as a means for evaluating the prediction performance of classifiers. However, I just can't understand this. I also checked ...
45
votes
2answers
63k views

Linear kernel and non-linear kernel for support vector machine?

When using support vector machine, are there any guidelines on choosing linear kernel vs. nonlinear kernel, like RBF? I once heard that non-linear kernel tends not to perform well once the number of ...
30
votes
2answers
2k views

What is the root cause of the class imbalance problem?

I've been thinking a lot about the "class imbalance problem" in machine/statistical learning lately, and am drawing ever deeper into a feeling that I just don't understand what is going on. First let ...
27
votes
5answers
8k views

What can cause PCA to worsen results of a classifier?

I have a classifier that I'm doing cross-validation on, along with a hundred or so features that I'm doing forward selection on to find optimal combinations of features. I also compare this against ...
25
votes
1answer
7k views

Is there any algorithm combining classification and regression?

I'm wondering if there's any algorithm could do classification and regression at the same time. For example, I'd like to let the algorithm learn a classifier, and at the same time within each label, ...
27
votes
3answers
12k views

What are the impacts of choosing different loss functions in classification to approximate 0-1 loss

We know that some objective functions are easier to optimize and some are hard. And there are many loss functions that we want to use but hard to use, for example 0-1 loss. So we find some proxy loss ...
26
votes
2answers
20k views

Three versions of discriminant analysis: differences and how to use them

Can anybody explain differences and give specific examples how to use these three analyses? LDA - Linear Discriminant Analysis FDA - Fisher's Discriminant Analysis QDA - Quadratic Discriminant ...
16
votes
1answer
566 views

What does it mean that AUC is a semi-proper scoring rule?

A proper scoring rule is a rule that is maximized by a 'true' model and it doesn't allow 'hedging' or gaming the system (deliberately reporting different results as is the true belief of the model to ...
72
votes
8answers
84k views

How to compute precision/recall for multiclass-multilabel classification?

I'm wondering how to calculate precision and recall measures for multiclass multilabel classification, i.e. classification where there are more than two labels, and where each instance can have ...
63
votes
4answers
85k views

Softmax vs Sigmoid function in Logistic classifier?

What decides the choice of function ( Softmax vs Sigmoid ) in a Logistic classifier ? Suppose there are 4 output classes . Each of the above function gives the probabilities of each class being the ...
60
votes
8answers
11k views

How can I help ensure testing data does not leak into training data?

Suppose we have someone building a predictive model, but that someone is not necessarily well-versed in proper statistical or machine learning principles. Maybe we are helping that person as they are ...
31
votes
2answers
42k views

Relative importance of a set of predictors in a random forests classification in R

I'd like to determine the relative importance of sets of variables toward a randomForest classification model in R. The ...
75
votes
4answers
39k views

Why are neural networks becoming deeper, but not wider?

In recent years, convolutional neural networks (or perhaps deep neural networks in general) have become deeper and deeper, with state-of-the-art networks going from 7 layers (AlexNet) to 1000 layers (...
24
votes
2answers
39k views

How large a training set is needed?

Is there a common method used to determine how many training samples are required to train a classifier (an LDA in this case) to obtain a minimum threshold generalization accuracy? I am asking ...
37
votes
3answers
44k views

SVM, Overfitting, curse of dimensionality

My dataset is small (120 samples), however the number of features are large varies from (1000-200,000). Although I'm doing feature selection to pick a subset of features, it might still overfit. My ...
18
votes
1answer
13k views

Opinions about Oversampling in general, and the SMOTE algorithm in particular [closed]

What is your opinion about oversampling in classification in general, and the SMOTE algorithm in particular? Why would we not just apply a cost/penalty to adjust for imbalance in class data and any ...
93
votes
3answers
133k views

How do you calculate precision and recall for multiclass classification using confusion matrix?

I wonder how to compute precision and recall using a confusion matrix for a multi-class classification problem. Specifically, an observation can only be assigned to its most probable class / label. I ...
36
votes
2answers
35k views

Logistic regression vs. LDA as two-class classifiers

I am trying to wrap my head around the statistical difference between Linear discriminant analysis and Logistic regression. Is my understanding right that, for a two class classification problem, LDA ...
15
votes
2answers
9k views

What measure of training error to report for Random Forests?

I'm currently fitting random forests for a classification problem using the randomForest package in R, and am unsure about how to report training error for these ...
11
votes
2answers
2k views

Compare classifiers based on AUROC or accuracy?

I have a binary classification problem and I experiment different classifiers on it: I want to compare the classifiers. which one is a better measure AUC or accuracy? And why? ...
27
votes
7answers
69k views

In Naive Bayes, why bother with Laplace smoothing when we have unknown words in the test set?

I was reading over Naive Bayes Classification today. I read, under the heading of Parameter Estimation with add 1 smoothing: Let $c$ refer to a class (such as Positive or Negative), and let $w$ ...
29
votes
4answers
62k views

Area under curve of ROC vs. overall accuracy

I am a little bit confusing about the Area Under Curve (AUC) of ROC and the overall accuracy. Will the AUC be proportional to the overall accuracy? In other words, when we have a larger overall ...
29
votes
2answers
25k views

How to statistically compare the performance of machine learning classifiers?

Based on estimated classification accuracy, I want to test whether one classifier is statistically better on a base set than another classifier . For each classifier, I select a training and testing ...
25
votes
4answers
18k views

When to avoid Random Forest?

Random forests are well known to perform fairly well on a variety of tasks and have been referred to as the leatherman of learning methods. Are there any types of problems or specific conditions in ...
17
votes
2answers
11k views

Testing Classification on Oversampled Imbalance Data

I am working on severely imbalanced data. In literature, several methods are used to re-balance the data using re-sampling (over- or under-sampling). Two good approaches are: SMOTE: Synthetic ...
16
votes
4answers
7k views

Is KNN a discriminative learning algorithm?

It seems that KNN is a discriminative learning algorithm but I can't seem to find any online sources confirming this. Is KNN a discriminative learning algorithm?
17
votes
3answers
16k views

Comparing two classifier accuracy results for statistical significance with t-test

I want to compare the accuracy of two classifiers for statistical significance. Both classifiers are run on the same data set. This leads me to believe I should be using a one sample t-test from what ...