Questions tagged [classification]

Statistical classification is the problem of identifying the sub-population to which new observations belong, where the identity of the sub-population is unknown, on the basis of a training set of data containing observations whose sub-population is known. Therefore these classifications will show a variable behavior which can be studied by statistics.

Filter by
Sorted by
Tagged with
16
votes
3answers
24k views

High Recall - Low Precision for unbalanced dataset

I’m currently encountering some problems analyzing a tweet dataset with support vector machines. The problem is that I have an unbalanced binary class training set (5:2); which is expected to be ...
16
votes
4answers
6k views

Checking whether accuracy improvement is significant

Suppose I have an algorithm that classifies things into two categories. I can measure the accuracy of the algorithm on say 1000 test things -- suppose 80% of the things are classified correctly. Lets ...
16
votes
1answer
12k views

The difference between logistic regression and support vector machines?

I know that logistic regression finds a hyperplane that separates the training samples. I also know that Support vector machines finds the hyperplane with the maximum margin. My question: is the ...
16
votes
2answers
12k views

training approaches for highly-imbalanced data set

I have a highly-imbalanced test data set. The positive set consists of 100 cases while the negative set consists of 1500 cases. On the training side, I have a larger candidate pool: the positive ...
16
votes
2answers
6k views

Mean(scores) vs Score(concatenation) in cross validation

TLDR: My dataset is pretty small (120) samples. While doing 10-fold cross validation, should I: Collect the outputs from each test fold, concatenate them into a vector, and then compute the error on ...
16
votes
1answer
1k views

Training a basic Markov Random Field for classifying pixels in an image

I am attempting to learn how to use Markov Random Fields to segment regions in an image. I do not understand some of the parameters in the MRF or why the expectation maximisation I perform fails to ...
15
votes
10answers
3k views

What is the best out-of-the-box 2-class classifier for your application? [closed]

Rules: one classifier per answer vote up if you agree downvote/remove duplicates. put your application in the comment
15
votes
5answers
55k views

How to interpret a ROC curve?

I applied logistic regression to my data on SAS and here are the ROC curve and classification table. I am comfortable with the figures in the classification table, but not exactly sure what the roc ...
15
votes
2answers
40k views

Using the caret package is it possible to obtain confusion matrices for specific threshold values?

I've obtained a logistic regression model (via train) for a binary response, and I've obtained the logistic confusion matrix via ...
15
votes
3answers
20k views

What should be the optimal parameters for Random Forest classifier?

Currently i am using RF toolbox on MATLAB for a binary classification Problem Data Set: 50000 samples and more than 250 features So what should be the number of trees and randomly selected feature ...
15
votes
1answer
3k views

Depth of a decision tree

Since the decision tree algorithm split on an attribute at every step, the maximum depth of a decision tree is equal to the number of attributes of the data. Is this correct?
15
votes
2answers
13k views

Mathematics behind classification and regression trees

Can anyone help explain some of the mathematics behind classification in CART? I'm looking to understand how two main stages happen. For instance I trained a CART classifier on a dataset and used a ...
15
votes
5answers
3k views

What is a good resource that includes a comparison of the pros and cons of different classifiers?

What is the best out-of-the-box 2-class classifier? Yes, I guess that's the million dollar question, and yes, I'm aware of the no free lunch theorem, and I've also read the previous questions: What ...
15
votes
3answers
2k views

Intuition for Support Vector Machines and the hyperplane

In my project I want to create a logistic regression model for predicting binary classification (1 or 0). I have 15 variables, 2 of which are categorical, while the rest are a mixture of continuous ...
15
votes
3answers
51k views

How to build a confusion matrix for a multiclass classifier?

I have a problem with 6 classes. So I build a multiclass classifier, as follows: for each class, I have one Logistic Regression classifier, using One vs. All, which means that I have 6 different ...
15
votes
2answers
14k views

Why Adaboost with Decision Trees?

I've been reading a bit on boosting algorithms for classification tasks and Adaboost in particular. I understand that the purpose of Adaboost is to take several "weak learners" and, through a set of ...
15
votes
3answers
9k views

PCA on high-dimensional text data before random forest classification?

Does it make sense to do PCA before carrying out a Random Forest Classification? I'm dealing with high dimensional text data, and I want to do feature reduction to help avoid the curse of ...
15
votes
2answers
31k views

Increasing number of features results in accuracy drop but prec/recall increase

I am new to Machine Learning. At the moment I am using a Naive Bayes (NB) classifier to classify small texts in 3 classes as positive, negative or neutral, using NLTK and python. After conducting ...
15
votes
5answers
3k views

Which statistical classification algorithm can predict true/false for a sequence of inputs?

Given a sequence of inputs, I need to determine whether this sequence has a certain desired property. The property can only be true or false, that is, there are only two possible classes that a ...
15
votes
1answer
2k views

Name of mean absolute error analogue to Brier score?

Yesterday's question Determine accuracy of model which estimates probability of event got me curious about probability scoring. The Brier score $$\frac{1}{N}\sum\limits _{i=1}^{N}(\text{prediction}_i -...
15
votes
4answers
194 views

Classification with tall fat data

I need to train a linear classifier on my laptop with hundreds of thousands of data points and about ten thousand features. What are my options? What is the state of the art for this type of problem? ...
15
votes
2answers
7k views

Why does the random forest OOB estimate of error improve when the number of features selected are decreased?

I am applying a random forest algorithm as a classifier on a microarray dataset which are split into two known groups with 1000s of features. After the initial run I look at the importance of the ...
15
votes
1answer
6k views

How does gradient boosting calculate probability estimates?

I have been trying to understand gradient boosting reading various blogs, websites and trying to find my answer by looking through for example the XGBoost source code. However, I cannot seem to find ...
15
votes
4answers
2k views

Statistical similarity of time series

Supposing one has a time series from which one can take various measurements such as period, maximum, minimum, average etc. and then use these to create a model sine wave with the same attributes, are ...
15
votes
2answers
1k views

Combining classifiers by flipping a coin

I am studying a machine learning course and the lecture slides contain information what I find contradicting with the recommended book. The problem is the following: there are three classifiers: ...
15
votes
4answers
8k views

What loss function should one use to get a high precision or high recall binary classifier?

I'm trying to make a detector of objects that occur very rarely (in images), planning to use a CNN binary classifier applied in a sliding/resized window. I've constructed balanced 1:1 positive-...
15
votes
3answers
8k views

How can machine learning models (GBM, NN etc.) be used for survival analysis?

I know that traditional statistical models like Cox Proportional Hazards regression & some Kaplan-Meier models can be used to predict days till next occurrence of an event say failure etc. i.e ...
15
votes
1answer
14k views

RandomForest - MDS plot interpretation

I used randomForest to classify 6 animal behaviours (eg. Standing, Walking, Swimming etc.) based on 8 variables (different body postures and movement). The MDSplot in the randomForest package gives ...
15
votes
4answers
8k views

Should one be concerned about multi-collinearity when using non-linear models?

Say we have a binary classification problem with mostly categorical features. We use some non-linear model (e.g. XGBoost or Random Forests) to learn it. Should one still be concerned about multi-...
14
votes
3answers
2k views

Can CART models be made robust?

A colleague in my office said to me today "Tree models aren't good because they get caught by extreme observations". A search here resulted in this thread that basically supports the claim. Which ...
14
votes
1answer
5k views

Comparison of two models when the ROC curves cross each other

One common measure used to compare two or more classification models is to use the area under the ROC curve (AUC) as a way to indirectly assess their performance. In this case a model with a larger ...
14
votes
3answers
15k views

Choice of neural net hidden activation function

I have read elsewhere that one's choice of hidden layer activation function in a NN should be based on one's need, i.e. if you need values in the range -1 to 1 use tanh and use sigmoid for the range 0 ...
14
votes
5answers
4k views

How to do one-class text classification?

I have to deal with a text classification problem. A web crawler crawls webpages of a certain domain and for each webpage I want to find out whether it belongs to only one specific class or not. That ...
14
votes
1answer
5k views

When over/under-sampling unbalanced classes, does maximizing accuracy differ from minimizing misclassification costs?

First of all, I would like to describe some common layouts that Data Mining books use explaining how to deal with Unbalanced Datasets. Usually the main section is named Unbalanced Datasets and they ...
14
votes
2answers
25k views

Is accuracy = 1- test error rate

Apologies if this is a very obvious question, but I have been reading various posts and can't seem to find a good confirmation. In the case of classification, is a classifier's accuracy = 1- test ...
14
votes
3answers
16k views

Outlier detection in very small sets

I need to get as accurate as possible a value for the brightness of a mainly stable light source given twelve sample luminosity values. The sensor is imperfect, and the light can occasionally "flicker"...
14
votes
1answer
7k views

Using LASSO on random forest

I would like to create a random forest using the following process: Build a tree on a random samples of the data and features using information gain to determine splits Terminate a leaf node if it ...
14
votes
3answers
10k views

Why does XGBoost have a learning rate?

Original Question Having used XGBoost a fair bit, clearly changing the learning rate dramatically affects the algorithm's performance. That said, I really can't understand the theoretical ...
14
votes
0answers
3k views

How can I get feature importance for Gaussian Naive Bayes classifier?

I have a dataset consisting of 4 classes and around 200 features. I have implemented a Gaussian Naive Bayes classifier. I want now calculate the importance of each feature for each pair of classes ...
13
votes
2answers
7k views

When is logistic regression suitable?

I'm currently teaching myself how to do classification, and specifically I'm looking at three methods: support vector machines, neural networks, and logistic regression. What I am trying to understand ...
13
votes
5answers
5k views

Why should binning be avoided at all costs?

So I've read a few posts about why binning should always be avoided. A popular reference for that claim being this link. The main getaway being that the binning points (or cutpoints) are rather ...
13
votes
2answers
20k views

Why is n-gram used in text language identification instead of words?

In two popular language identification libraries, Compact Language Detector 2 for C++ and language detector for java, both of them used (character based) n-grams to extract text features. Why is a bag-...
13
votes
3answers
28k views

What is a good AUC for a precision-recall curve?

Because I have a very imbalanced dataset (9% positive outcomes), I decided a precision-recall curve was more appropriate than an ROC curve. I obtained the analogous summary measure of area under the P-...
13
votes
2answers
3k views

Do CART trees capture interactions among predictors?

This paper claims that in CART, because a binary split is performed on a single covariate at each step, all splits are orthogonal and therefore interactions among covariates are not considered. ...
13
votes
3answers
7k views

Restricted Boltzmann Machines for regression?

I'm following up on the question I'd asked earlier on RBMs. I see a lot of literature describing them but none that actually talks of regression (not even classification with labelled data). I get a ...
13
votes
2answers
5k views

Classification with ordered classes?

Say I want to train a classifier that assigns an image of a person as young, middle-aged, or old. A simple way would be to treat the classes as independent categories and train a classifier. But ...
13
votes
2answers
11k views

How do sample weights work in classification models?

What does it mean to provide weights to each sample in a classification algorithm? How does a classification algorithm (eg. Logistic regression, SVM) use weights to give more emphasis to certain ...
13
votes
1answer
716 views

Proper scoring rule when there is a decision to make (e.g. spam vs ham email)

Among others on here, Frank Harrell is adamant about using proper scoring rules to assess classifiers. This makes sense. If we have 500 $0$s with $P(1)\in[0.45, 0.49]$ and 500 $1$s with $P(1)\in[0.51, ...
13
votes
2answers
19k views

How can a multiclass perceptron work?

I don't have any background in math, but I understand how the simple Perceptron works and I think I grasp the concept of a hyperplane (I imagine it geometrically as a plane in 3D space which seperates ...
13
votes
1answer
24k views

How to train LSTM layer of deep-network

I'm using a lstm and feed-forward network to classify text. I convert the text into one-hot vectors and feed each into the lstm so I can summarise it as a single representation. Then I feed it to the ...