Questions tagged [unbalanced-classes]

Data organized into discrete categories or *classes* may present problems for certain analyses if the number of observations ($n$) belonging to each class is not constant across classes. Classes with unequal $n$ are *unbalanced*.

Filter by
Sorted by
Tagged with
100
votes
3answers
98k views

Does an unbalanced sample matter when doing logistic regression?

Okay, so I think I have a decent enough sample, taking into account the 20:1 rule of thumb: a fairly large sample (N=374) for a total of 7 candidate predictor variables. My problem is the following: ...
77
votes
7answers
21k views

When is unbalanced data really a problem in Machine Learning?

We already had multiple questions about unbalanced data when using logistic regression, SVM, decision trees, bagging and a number of other similar questions, what makes it a very popular topic! ...
60
votes
7answers
36k views

Binary classification with strongly unbalanced classes

I have a data set in the form of (features, binary output 0 or 1), but 1 happens pretty rarely, so just by always predicting 0, I get accuracy between 70% and 90% (depending on the particular data I ...
52
votes
5answers
52k views

Training a decision tree against unbalanced data

I'm new to data mining and I'm trying to train a decision tree against a data set which is highly unbalanced. However, I'm having problems with poor predictive accuracy. The data consists of students ...
50
votes
4answers
26k views

Class imbalance in Supervised Machine Learning

This is a question in general, not specific to any method or data set. How do we deal with a class imbalance problem in Supervised Machine learning where the number of 0 is around 90% and number of 1 ...
47
votes
4answers
79k views

What is the proper usage of scale_pos_weight in xgboost for imbalanced datasets?

I have a very imbalanced dataset. I'm trying to follow the tuning advice and use scale_pos_weight but not sure how should I tune it. I can see that ...
43
votes
3answers
4k views

What is the root cause of the class imbalance problem?

I've been thinking a lot about the "class imbalance problem" in machine/statistical learning lately, and am drawing ever deeper into a feeling that I just don't understand what is going on. ...
39
votes
4answers
38k views

When should I balance classes in a training data set?

I had an online course, where I learned, that unbalanced classes in the training data might lead to problems, because classification algorithms go for the majority rule, as it gives good results if ...
38
votes
1answer
10k views

Does down-sampling change logistic regression coefficients?

If I have a dataset with a very rare positive class, and I down-sample the negative class, then perform a logistic regression, do I need to adjust the regression coefficients to reflect the fact that ...
37
votes
0answers
1k views

Are unbalanced datasets problematic, and (how) does oversampling (purport to) help? [duplicate]

TL;DR See title. Motivation I am hoping for a canonical answer along the lines of "(1) No, (2) Not applicable, because (1)", which we can use to close many wrong questions about unbalanced datasets ...
33
votes
6answers
24k views

Sampling for Imbalanced Data in Regression

There have been good questions on handling imbalanced data in the classification context, but I am wondering what people do to sample for regression. Say the problem domain is very sensitive to the ...
32
votes
4answers
15k views

Optimising for Precision-Recall curves under class imbalance

I have a classification task where I have a number of predictors (one of which is the most informative), and I am using the MARS model to construct my classifier (I am interested in any simple model, ...
31
votes
3answers
31k views

Classification/evaluation metrics for highly imbalanced data

I deal with a fraud detection (credit-scoring-like) problem. As such there is a highly imbalanced relation between fraudulent and non-fraudulent observations. http://blog.revolutionanalytics.com/2016/...
30
votes
6answers
98k views

Sample size for logistic regression?

I want to make a logistic model from my survey data. It is a small survey of four residential colonies in which only 154 respondents were interviewed. My dependent variable is "satisfactory transition ...
30
votes
4answers
7k views

What problem does oversampling, undersampling, and SMOTE solve?

In a recent, well recieved, question, Tim asks when is unbalanced data really a problem in Machine Learning? The premise of the question is that there is a lot of machine learning literature ...
26
votes
3answers
16k views

ROC vs Precision-recall curves on imbalanced dataset

I just finished reading this discussion. They argue that PR AUC is better than ROC AUC on imbalanced dataset. For example, we have 10 samples in test dataset. 9 samples are positive and 1 is ...
24
votes
2answers
19k views

Testing Classification on Oversampled Imbalance Data

I am working on severely imbalanced data. In literature, several methods are used to re-balance the data using re-sampling (over- or under-sampling). Two good approaches are: SMOTE: Synthetic ...
24
votes
2answers
48k views

Adding weights to logistic regression for imbalanced data

I want to model a logistic regression with imbalanced data (9:1). I wanted to try the weights option in the glm function in R, but I'm not 100% sure what it does. ...
24
votes
2answers
8k views

How to handle the difference between the distribution of the test set and the training set?

I think one basic assumption of machine learning or parameter estimation is that the unseen data come from the same distribution as the training set. However, in some practical cases, the distribution ...
23
votes
2answers
17k views

The order of variables in ANOVA matters, doesn't it?

Am I correct to understand that the order in which variables are specified in a multifactorial ANOVA makes a difference but that the order does not matter when doing a multiple linear regression? So ...
22
votes
1answer
18k views

Balanced accuracy vs F-1 score

I was wondering if anyone could explain the difference between balanced accuracy which is b_acc = (sensitivity + specificity)/2 and f1 score which is: ...
18
votes
3answers
25k views

SVM for unbalanced data

I want to attempt to use Support Vector Machines (SVMs) on my dataset. Before I attempt the problem though, I was warned that SVMs dont perform well on extremely unbalanced data. In my case, I can ...
18
votes
2answers
6k views

Does GBM classification suffer from imbalanced class sizes?

I'm dealing with a supervised binary classification issue. I'd like to use the GBM package to classify individuals as uninfected/infected. I have 15 times more uninfected than infected individuals. I ...
17
votes
2answers
11k views

Why P>0.5 cutoff is not “optimal” for logistic regression?

PREFACE: I don't care about the merits of using a cutoff or not, or how one should choose a cutoff. My question is purely mathematical and due to curiosity. Logistic regression models the posterior ...
17
votes
3answers
29k views

High Recall - Low Precision for unbalanced dataset

I’m currently encountering some problems analyzing a tweet dataset with support vector machines. The problem is that I have an unbalanced binary class training set (5:2); which is expected to be ...
16
votes
3answers
10k views

Suggestions for cost-sensitive learning in a highly imbalanced setting

I have a dataset with a few million rows and ~100 columns. I would like to detect about 1% of the examples in the dataset, which belong to a common class. I have a minimum precision constraint, but ...
16
votes
1answer
12k views

Is gradient boosting appropriate for data with low event rates like 1%?

I am trying gradient boosting on a dataset with event rate about 1% using Enterprise miner, but it is failing to produce any output. My question is, since it a decision tree based approach, is it even ...
15
votes
4answers
9k views

What loss function should one use to get a high precision or high recall binary classifier?

I'm trying to make a detector of objects that occur very rarely (in images), planning to use a CNN binary classifier applied in a sliding/resized window. I've constructed balanced 1:1 positive-...
14
votes
2answers
14k views

Does Support Vector Machine handle imbalanced Dataset?

Does SVM handles imbalanced dataset? Is that any parameters (like C, or misclassification cost) handling the imbalanced dataset?
14
votes
1answer
5k views

When over/under-sampling unbalanced classes, does maximizing accuracy differ from minimizing misclassification costs?

First of all, I would like to describe some common layouts that Data Mining books use explaining how to deal with Unbalanced Datasets. Usually the main section is named Unbalanced Datasets and they ...
13
votes
2answers
599 views

Brier Score and extreme class imbalance

Since I've heard about proper scoring rules for binary classification like the Brier score or Log Loss, I am more and more convinced that they are drastically underrepresented in practice in favor of ...
12
votes
1answer
17k views

ROSE and SMOTE oversampling methods

Can somebody give me a brief explanation of the differences between those two resampling methods : ROSE and SMOTE ?
12
votes
1answer
398 views

Creating an Imbalanced Dataset

I would like to have my trained model tested on an imbalanced dataset. Is there any algorithms available to generate synthetic data from a balanced labelled dataset (spam/non-spam)?
12
votes
1answer
17k views

How to reduce number of false positives?

I'm trying to solve task called pedestrian detection and I train binary clasifer on two categories positives - people, negatives - background. I have dataset: number of positives= 3752 number of ...
12
votes
2answers
1k views

Is up- or down-sampling imbalanced data actually that effective? Why?

I frequently hear up- or down-sampling of data discussed as a way of dealing with classification of imbalanced data. I understand that this could be useful if you're working with a binary (as opposed ...
11
votes
2answers
12k views

Is f-measure synonymous with accuracy?

I understand that f-measure (based on precision and recall) is an estimate of how accurate a classifier is. Also, f-measure is favored over accuracy when we have an unbalanced dataset. I have a simple ...
11
votes
1answer
13k views

Oversampling with categorical variables

I would like to perform a combination of oversampling and undersampling in order to balance my dataset with roughly 4000 customers divided into two groups, where one of the groups have a proportion of ...
10
votes
6answers
31k views

Logistic regression is predicting all 1, and no 0

I am running an analysis on the probability of loan default using logistic regression and random forests. When I use logistic regression, the prediction is always all '1' (which means good loan). ...
10
votes
1answer
8k views

ROC curves for unbalanced datasets

Consider an input matrix $X$ and a binary output $y$. A common way to measure the performance of a classifier is to use ROC curves. In a ROC plot the diagonal is the result that would be obtained ...
10
votes
3answers
5k views

Training data is imbalanced - but should my validation set also be?

I have labelled data composed of 10000 positive examples, and 50000 negative examples, giving a total of 60000 examples. Obviously this data is imbalanced. Now let us say I want to create my ...
10
votes
1answer
8k views

SMOTE throws error for multi class imbalance problem

I am trying to use SMOTE to correct imbalance in my multi-class classification problem. Although SMOTE works perfectly on the iris dataset as per the SMOTE help document, it does not work on a similar ...
10
votes
1answer
11k views

classification threshold in RandomForest-sklearn

1) How can I change classification threshold (i think it is 0.5 by default) in RandomForest in sklearn? 2) how can I under-sample in sklearn? 3) I have the following result from RandomForest ...
10
votes
1answer
1k views

Softmax regression bias and prior probabilities for unequal classes

I'm using Softmax regression for a multi-class classification problem. I don't have equal prior probabilities for each of the classes. I know from Logistic Regression (softmax regression with 2 ...
9
votes
3answers
3k views

Experiment design: Can unbalanced dataset be better than balanced?

I'm on the stage of experiment design of some biomedical time-course study. Let's say we will have 2 groups of subjects - case and control. The total number of subjects is limited (for example, 30), ...
9
votes
2answers
5k views

Does a big difference in sample sizes together with a difference in variances matter for a t-test (or permutation test)?

There is a very confusing question in my mind. I have data, and would like to compare numeric scores between men and women. There is a big difference in those two groups: the number of men is 34, ...
9
votes
2answers
1k views

Why is a PR curve considered better than an ROC curve for imbalanced datasets?

I have heard from multiple sources that a precision-recall curve is considered better than an ROC curve when testing a classifier on a dataset with a class imbalance. https://www.biostat.wisc.edu/~...
9
votes
1answer
2k views

Are unequal groups a problem for one-way ANOVA?

I have data for three unequal groups: $N = 44$, $N = 354$ and $N = 347$. Is it possible to compare all three groups running a one-way ANOVA or is the first group too small?
9
votes
2answers
3k views

Named entity recognition and class imbalance [duplicate]

I have implemented Maximum-entropy Markov model (MEMM) for the Named entity recognition (NER) problem. I have four classes: geographical, people, material (book titles etc) and other. Class ...
9
votes
1answer
5k views

A priori selection of SVM class weights

I remember seeing/reading somewhere that for multiclass SVMs with unbalanced data, there was a way to determine the class weights from the training data (rather than X validation). Does anyone know ...
9
votes
1answer
7k views

Best way to handle unbalanced multiclass dataset with SVM

I'm trying to build a prediction model with SVMs on fairly unbalanced data. My labels/output have three classes, positive, neutral and negative. I would say the positive example makes about 10 - 20% ...

1
2 3 4 5
18