Questions tagged [classification]

Statistical classification is the problem of identifying the sub-population to which new observations belong, where the identity of the sub-population is unknown, on the basis of a training set of data containing observations whose sub-population is known. Therefore these classifications will show a variable behavior which can be studied by statistics.

Filter by
Sorted by
Tagged with
29
votes
7answers
4k views

What are the branches of statistics?

In mathematics, there are branches such as algebra, analysis, topology, etc. In machine learning there is supervised, unsupervised, and reinforcement learning. Within each of these branches, there are ...
29
votes
4answers
62k views

Area under curve of ROC vs. overall accuracy

I am a little bit confusing about the Area Under Curve (AUC) of ROC and the overall accuracy. Will the AUC be proportional to the overall accuracy? In other words, when we have a larger overall ...
29
votes
2answers
60k views

Difference between naive Bayes & multinomial naive Bayes

I've dealt with Naive Bayes classifier before. I've been reading about Multinomial Naive Bayes lately. Also Posterior Probability = (Prior * Likelihood)/(Evidence). The only prime difference (while ...
29
votes
4answers
22k views

When should I balance classes in a training data set?

I had an online course, where I learned, that unbalanced classes in the training data might lead to problems, because classification algorithms go for the majority rule, as it gives good results if ...
29
votes
3answers
26k views

Why is AUC higher for a classifier that is less accurate than for one that is more accurate?

I have two classifiers A: naive Bayesian network B: tree (singly-connected) Bayesian network In terms of accuracy and other measures, A performs comparatively worse than B. However, when I use the R ...
29
votes
6answers
13k views

Variable selection procedure for binary classification

What are the variable/feature selection that you prefer for binary classification when there are many more variables/feature than observations in the learning set? The aim here is to discuss what is ...
29
votes
2answers
25k views

How to statistically compare the performance of machine learning classifiers?

Based on estimated classification accuracy, I want to test whether one classifier is statistically better on a base set than another classifier . For each classifier, I select a training and testing ...
27
votes
5answers
8k views

What can cause PCA to worsen results of a classifier?

I have a classifier that I'm doing cross-validation on, along with a hundred or so features that I'm doing forward selection on to find optimal combinations of features. I also compare this against ...
27
votes
4answers
53k views

How to measure/rank “variable importance” when using CART? (specifically using {rpart} from R)

When building a CART model (specifically classification tree) using rpart (in R), it is often interesting to know what is the importance of the various variables introduced to the model. Thus, my ...
27
votes
7answers
70k views

In Naive Bayes, why bother with Laplace smoothing when we have unknown words in the test set?

I was reading over Naive Bayes Classification today. I read, under the heading of Parameter Estimation with add 1 smoothing: Let $c$ refer to a class (such as Positive or Negative), and let $w$ ...
27
votes
2answers
1k views

When is it appropriate to use an improper scoring rule?

Merkle & Steyvers (2013) write: To formally define a proper scoring rule, let $f$ be a probabilistic forecast of a Bernoulli trial $d$ with true success probability $p$. Proper scoring ...
27
votes
3answers
12k views

What are the impacts of choosing different loss functions in classification to approximate 0-1 loss

We know that some objective functions are easier to optimize and some are hard. And there are many loss functions that we want to use but hard to use, for example 0-1 loss. So we find some proxy loss ...
27
votes
1answer
8k views

Is cross validation a proper substitute for validation set?

In text classification, I have a training set with about 800 samples, and a test set with about 150 samples. The test set has never been used, and waiting to be used until the end. I am using the ...
26
votes
1answer
42k views

One-vs-All and One-vs-One in svm?

What is the difference between a one-vs-all and a one-vs-one SVM classifier? Does the one-vs-all mean one classifier to classify all types / categories of the new image and one-vs-one mean each type /...
26
votes
2answers
20k views

Three versions of discriminant analysis: differences and how to use them

Can anybody explain differences and give specific examples how to use these three analyses? LDA - Linear Discriminant Analysis FDA - Fisher's Discriminant Analysis QDA - Quadratic Discriminant ...
26
votes
4answers
54k views

Predicting with both continuous and categorical features

Some predictive modeling techniques are more designed for handling continuous predictors, while others are better for handling categorical or discrete variables. Of course there exist techniques to ...
25
votes
5answers
4k views

How can top principal components retain the predictive power on a dependent variable (or even lead to better predictions)?

Suppose I am running a regression $Y \sim X$. Why by selecting top $k$ principle components of $X$, does the model retain its predictive power on $Y$? I understand that from dimensionality-reduction/...
25
votes
4answers
18k views

When to avoid Random Forest?

Random forests are well known to perform fairly well on a variety of tasks and have been referred to as the leatherman of learning methods. Are there any types of problems or specific conditions in ...
25
votes
2answers
16k views

Does it make sense to combine PCA and LDA?

Assume I have a dataset for a supervised statistical classification task, e.g., via a Bayes' classifier. This dataset consists of 20 features and I want to boil it down to 2 features via ...
25
votes
1answer
7k views

Is there any algorithm combining classification and regression?

I'm wondering if there's any algorithm could do classification and regression at the same time. For example, I'd like to let the algorithm learn a classifier, and at the same time within each label, ...
25
votes
3answers
13k views

Top five classifiers to try first

Besides obvious classifier characteristics like computational cost, expected data types of features/labels and suitability for certain sizes and dimensions of data sets, what are the top five (or 10,...
25
votes
2answers
24k views

Neural Network: For Binary Classification use 1 or 2 output neurons?

Assume I want to do binary classification (something belongs to class A or class B). There are some possibilities to do this in the output layer of a neural network: Use 1 output node. Output 0 (<...
25
votes
2answers
1k views

Detecting patterns of cheating on a multi-question exam

QUESTION: I have binary data on exam questions (correct/incorrect). Some individuals might have had prior access to a subset of questions and their correct answers. I don’t know who, how many, or ...
24
votes
3answers
16k views

Cross-validation or bootstrapping to evaluate classification performance?

What is the most appropriate sampling method to evaluate the performance of a classifier on a particular data set and compare it with other classifiers? Cross-validation seems to be standard practice, ...
24
votes
2answers
39k views

How large a training set is needed?

Is there a common method used to determine how many training samples are required to train a classifier (an LDA in this case) to obtain a minimum threshold generalization accuracy? I am asking ...
24
votes
2answers
16k views

Bag-of-Words for Text Classification: Why not just use word frequencies instead of TFIDF?

A common approach to text classification is to train a classifier off of a 'bag-of-words'. The user takes the text to be classified and counts the frequencies of the words in each object, followed by ...
24
votes
3answers
10k views

What are the advantages of stacking multiple LSTMs?

What are the advantages, why would one use multiple LSTMs, stacked one side-by-side, in a deep-network? I am using a LSTM to represent a sequence of inputs as a single input. So once I have that ...
24
votes
1answer
412 views

What is the probability that $n$ random points in $d$ dimensions are linearly separable?

Given $n$ data points, each with $d$ features, $n/2$ are labeled as $0$, the other $n/2$ are labeled as $1$. Each feature takes a value from $[0,1]$ randomly (uniform distribution). What's the ...
23
votes
2answers
16k views

What is the difference between a loss function and decision function?

I see that both functions are part of data mining methods such as Gradient Boosting Regressors. I see that those are separate objects too. How is the relationship between both in general?
23
votes
4answers
33k views

Why do researchers use 10-fold cross validation instead of testing on a validation set?

I have read a lot of research papers about sentiment classification and related topics. Most of them use 10-fold cross validation to train and test classifiers. That means that no separate testing/...
23
votes
5answers
7k views

Alternatives to classification trees, with better predictive (e.g: CV) performance?

I am looking for an alternative to Classification Trees which might yield better predictive power. The data I am dealing with has factors for both the explanatory and the explained variables. I ...
23
votes
3answers
5k views

Visualizing the calibration of predicted probability of a model

Suppose I have a predictive model that produces, for each instance, a probability for each class. Now I recognize that there are many ways to evaluate such a model if I want to use those ...
23
votes
2answers
7k views

How to handle the difference between the distribution of the test set and the training set?

I think one basic assumption of machine learning or parameter estimation is that the unseen data come from the same distribution as the training set. However, in some practical cases, the distribution ...
22
votes
6answers
50k views

Test accuracy higher than training. How to interpret?

I've a dataset containing at most 150 examples (split into training & test), with many features (higher than 1000). I need to compare classifiers and feature selection methods which perform well ...
22
votes
3answers
23k views

Classification/evaluation metrics for highly imbalanced data

I deal with a fraud detection (credit-scoring-like) problem. As such there is a highly imbalanced relation between fraudulent and non-fraudulent observations. http://blog.revolutionanalytics.com/2016/...
22
votes
2answers
10k views

Restricted Boltzmann machines vs multilayer neural networks

I've been wanting to experiment with a neural network for a classification problem that I'm facing. I ran into papers that talk of RBMs. But from what I can understand, they are no different from ...
22
votes
4answers
2k views

When are Shao's results on leave-one-out cross-validation applicable?

In his paper Linear Model Selection by Cross-Validation, Jun Shao shows that for the problem of variable selection in multivariate linear regression, the method of leave-one-out cross validation (...
22
votes
3answers
39k views

Supervised clustering or classification?

The second question is that I found in a discussion somewhere on the web talking about "supervised clustering", as far as I know, clustering is unsupervised, so what is exactly the meaning behind "...
22
votes
1answer
3k views

Choosing among proper scoring rules

Most resources on proper scoring rules mention a number of different scoring rules like log-loss, Brier score or spherical scoring. However, they often don't give much guidance on the differences ...
22
votes
3answers
15k views

interpreting y axis of a partial dependence plots

I have read through other topics on partial dependence plots and most of them are on how you actually plot them with different packages, not how you can accurately interpret them, So: I have been ...
21
votes
2answers
22k views

Convolutional neural network for time series?

I would like to know if there exists a code to train a convolutional neural net to do time-series classification. I have seen some recent papers (http://www.fer.unizg.hr/_download/repository/KDI-...
21
votes
2answers
34k views

Adding weights to logistic regression for imbalanced data

I want to model a logistic regression with imbalanced data (9:1). I wanted to try the weights option in the glm function in R, but I'm not 100% sure what it does. ...
21
votes
5answers
8k views

What's the correct way to test the significance of classification results

There are many situations where you may train several different classifiers, or use several different feature extraction methods. In the literature authors often give the mean classification error ...
21
votes
5answers
12k views

How to control the cost of misclassification in Random Forests?

Is it possible to control the cost of misclassification in the R package randomForest? In my own work false negatives (e.g., missing in error that a person may have a disease) are far more costly ...
21
votes
4answers
3k views

Why does the least square solution give poor results in this case?

There is an image in page 204, chapter 4 of "pattern recognition and machine learning" by Bishop where I don't understand why the Least square solution gives poor results here: The previous paragraph ...
21
votes
3answers
12k views

From the Perceptron rule to Gradient Descent: How are Perceptrons with a sigmoid activation function different from Logistic Regression?

Essentially, my question is that in multilayer Perceptrons, perceptrons are used with a sigmoid activation function. So that in the update rule $\hat{y}$ is calculated as $$\hat{y} = \frac{1}{1+\exp(...
20
votes
3answers
15k views

Test for linear separability

Is there a way to test linear separability of a two-class dataset in high dimensions? My feature vectors are 40-long. I know I can always run logistic regression experiments and determine hitrate vs ...
20
votes
4answers
3k views

Summary of “Large p, Small n” results

Can anybody point me to a survey paper on "Large $p$, Small $n$" results? I am interested in how this problem manifests itself in different research contexts, e.g. regression, classification, ...
20
votes
3answers
24k views

Machine Learning to Predict Class Probabilities

I am looking for classifiers that output probabilties that examples belong to one of two classes. I know of logistic regression and naive Bayes, but can you tell me of others that work in a similar ...
20
votes
2answers
3k views

What problem does oversampling, undersampling, and SMOTE solve?

In a recent, well recieved, question, Tim asks when is unbalanced data really a problem in Machine Learning? The premise of the question is that there is a lot of machine learning literature ...