Data organized into discrete categories or *classes* may present problems for certain analyses if the number of observations ($n$) belonging to each class is not constant across classes. Classes with unequal $n$ are *unbalanced*.

learn more… | top users | synonyms

12
votes
1answer
869 views

How to handle the difference between the distribution of the test set and the training set?

I think one basic assumption of machine learning or parameter estimation is that the unseen data come from the same distribution as the training set. However, in some practical cases, the distribution ...
12
votes
1answer
2k views

Optimising for Precision-Recall curves under class imbalance

I have a classification task where I have a number of predictors (one of which is the most informative), and I am using the MARS model to construct my classifier (I am interested in any simple model, ...
11
votes
2answers
2k views

Does GBM classification suffer from imbalanced class sizes?

I'm dealing with a supervised binary classification issue. I'd like to use the GBM package to classify individuals as uninfected/infected. I have 15 times more uninfected than infected individuals. I ...
10
votes
2answers
5k views

Training a decision tree against unbalanced data

I'm new to data mining and I'm trying to train a decision tree against a data set which is highly unbalanced. However, I'm having problems with poor predictive accuracy. The data consists of students ...
9
votes
1answer
2k views

When over/under-sampling unbalanced classes, does maximizing accuracy differ from minimizing misclassification costs?

First of all, I would like to describe some common layouts that Data Mining books use explaining how to deal with Unbalanced Datasets. Usually the main section is named Unbalanced Datasets and they ...
8
votes
2answers
1k views

Does a big difference in sample sizes matter for an independent t-test?

There is a very confusing question in my mind. I have data, and would like to compare numeric scores between men and women. There is a big difference in those two groups: the number of men is 34, ...
7
votes
1answer
1k views

A priori selection of SVM class weights

I remember seeing/reading somewhere that for multiclass SVMs with unbalanced data, there was a way to determine the class weights from the training data (rather than X validation). Does anyone know ...
7
votes
2answers
2k views

Naive-Bayes classifier for unequal groups

I'm using naive bayes classifier to classify between two groups of data. One group of the data is much larger than the other (above 4 times). I'm using the prior probability of each group in the ...
6
votes
1answer
4k views

Best way to handle unbalanced multiclass dataset with SVM

I'm trying to build a prediction model with SVMs on fairly unbalanced data. My labels/output have three classes, positive, neutral and negative. I would say the positive example makes about 10 - 20% ...
6
votes
1answer
624 views

CART (rpart) balanced vs. unbalanced dataset

I am fitting a tree (CART) to the olives-dataset. The training data has 436 observations (test data: 136). I have 3 responses (the 'Region' variable) which splits the training data into 116 / 74 / 246 ...
5
votes
3answers
421 views

Experiment design: Can unbalanced dataset be better than balanced?

I'm on the stage of experiment design of some biomedical time-course study. Let's say we will have 2 groups of subjects - case and control. The total number of subjects is limited (for example, 30), ...
5
votes
2answers
557 views

Feature selection for low probability event prediction

I'm currently trying to predict the probability for low probability events (~1%). I have large DB with ~200,000 vectors (~2000 plus examples) with ~200 features. I'm trying to find the the best ...
5
votes
3answers
792 views

Might be an unbalanced within subjects repeated measures?

I ran a within subjects repeated measures experiment, where the independent variable had 3 levels. The dependent variable is a measure of correctness and is recorded as either correct / incorrect. ...
5
votes
2answers
898 views

Testing Classification on Oversampled Imbalance Data

I am working on severely imbalanced data. In literature, several methods are used to re-balance the data using re-sampling (over- or under-sampling). Two good approaches are: SMOTE: Synthetic ...
5
votes
2answers
2k views

GLM on unbalanced design

I have a dataset that comprises 200 males and 250 females and I am testing their responses on the relationship between X and Y. X and Y are continuous and X1 (gender) is categorical. I am using ...
5
votes
2answers
799 views

Classification problem using imbalanced dataset

I am working on a pattern identification/classification problem on an imbalanced dataset, with target to non target proportion in population approx as 1%:99%. There are around 0.5 million records in ...
5
votes
1answer
780 views

classification threshold in RandomForest-sklearn

1) How can I change classification threshold (i think it is 0.5 by default) in RandomForest in sklearn? 2) how can I under-sample in sklearn? 3) I have the following result from RandomForest ...
4
votes
3answers
2k views

SVM vs. artificial neural network

I have multiclass unbalanced data (4 class with 15% 25% 45% 15% data in each class). Which method is good for classification of such data- SVM or ANN? UPDATE- Let me make the question little more ...
4
votes
2answers
356 views

Does Support Vector Machine handle imbalanced Dataset?

Does SVM handles imbalanced dataset? Is that any parameters (like C, or misclassification cost) handling the imbalanced dataset?
4
votes
2answers
4k views

Calculating statistical significance with unequal sample sizes and unequal variances

I have two samples, one with $n_1 = 41,000$ and the other with $n_2 = 881$; the larger sample has a standard deviation of $13.74$, and the smaller has an $SD=10.75$. The means are different, and when ...
4
votes
3answers
2k views

SVM for unbalanced data

I want to attempt to use Support Vector Machines (SVMs) on my dataset. Before I attempt the problem though, I was warned that SVMs dont perform well on extremely unbalanced data. In my case, I can ...
4
votes
1answer
892 views

Is a large control sample better than a balanced sample size when the treatment group is small?

I am running an experiment looking at brain volume changes in a rare disorder. We have a small number of patients (n = 8) but a large control group (n = 100). Some colleagues have suggested that a ...
4
votes
1answer
117 views

Logistic regression without negative samples

I have a data set of RNA reaction values of breast cancer. I want to figure out which RNAs are essential genes by Logistic Regression & LASSO. The data set has no negative samples. What should I ...
4
votes
2answers
156 views

Finding occurrences of specific patterns in time series

I have to locate occurrences of Cyllinder, Bell and Funnel patterns in univariate time series $X$ of gamma-ray sensoring. This is a specific case of the general CBF synthetic problem found in a few ...
4
votes
1answer
96 views

Is up- or down-sampling imbalanced data actually that effective? Why?

I frequently hear up- or down-sampling of data discussed as a way of dealing with classification of imbalanced data. I understand that this could be useful if you're working with a binary (as opposed ...
4
votes
0answers
469 views

Mixed-effects model for a strongly unbalanced design

I am somehow unsure on the best option to analyze these data. Here is my study case: The response variable is a morphometric measure, one for each individual. During 10 years, say 2000-2009, people ...
4
votes
0answers
333 views

Softmax regression bias and prior probabilities for unequal classes

I'm using Softmax regression for a multi-class classification problem. I don't have equal prior probabilities for each of the classes. I know from Logistic Regression (softmax regression with 2 ...
3
votes
3answers
1k views

Class imbalance in Supervised Machine Learning

This is a question in general, not specific to any method or data set. How do we deal with a class imbalance problem in Supervised Machine learning where the number of 0 is around 90% and number of 1 ...
3
votes
1answer
354 views

Using Wilcoxon-Mann-Whitney test for comparing two population of different sizes

I am using Wilcoxon-Mann-Whitney test for comparing two populations. Unfortunately, sizes of my population are different; one has size 100000 and the other 6000. Can I use this test to compare these ...
3
votes
2answers
340 views

Named entity recognition and class imbalance

I have implemented Maximum-entropy Markov model (MEMM) for the Named entity recognition (NER) problem. I have four classes: geographical, people, material (book titles etc) and other. Class ...
3
votes
2answers
2k views

By using SMOTE the classification of the validation set is bad

I want to do classification with 2 classes. When I classify without smote I get: ...
3
votes
1answer
2k views

Balanced accuracy vs F-1 score

I was wondering if anyone could explain the difference between balanced accuracy which is b_acc = (sensitivity + specificity)/2 and f1 score which is: ...
3
votes
2answers
5k views

LibSVM cost weights for unbalanced data doesn't work

I have a dataset where the number of negative labeled values is 163 times the number of positive labeled values. That is, I have an unbalanced data set. I tried: ...
3
votes
1answer
577 views

Are there problems with inference using linear regression on observational data with highly skewed distributions of predictor values?

I am using a linear regression model to perform inference on some observational data. The samples are from an observational study and highly skewed along some of the dummy variables in the regression. ...
3
votes
1answer
56 views

What's the measure to assess the binary classification accuracy for imbalanced data?

Now I have binary classification problem with positive samples roughly 100 times the number of negative samples. In this case the normal accuracy measure (predict == label) is not a good measure. What ...
3
votes
1answer
217 views

How to reduce number of false positives?

I'm trying to solve task called pedestrian detection and I train binary clasifer on two categories positives - people, negatives - background. I have dataset: number of positives= 3752 number of ...
3
votes
1answer
2k views

Which performance measure for unbalanced binary classification without an 'active' class?

My datasets have two classes A and B. The classes should be treated equally (there is no "active/inactive"). The datasets are unbalanced, sometimes A is more frequent, sometimes B is more frequent. ...
3
votes
1answer
1k views

Using AdaBoost on multi-class in R on unbalanced data

I have a data set which is highly imbalanced and I have used the SMOTE algorithm (using the R package DMwR) to balance the binary class in the data set. I have been using the R Ada package to then ...
3
votes
1answer
160 views

Cross Validation in Unbalanced Datasets

Is there a specific way of sampling which maintains the ratio of samples in an unbiased set? e.g., lets say I want to do k-fold cross-validation on my training set And my training set is very ...
3
votes
1answer
63 views

What test of significance should I use to analyse visits to a clinic by various groups?

Apologies for the noob question. After a day on Google and Wikipedia I still can't quite work out what to do. So here I am. A small private healthcare clinic in the UK has asked me to look at their ...
3
votes
0answers
53 views

Bias Correction for Large Scale Logistic Regression with Rare Events

I have a large dataset constituted of many ad impressions. My dependent binary variable clicked describe whether or not the ad was clicked on. As you can expect, the number of clicks is about 1000x ...
3
votes
0answers
227 views

Mixed ANOVA: small and unbalanced samples

I have to analyze two samples ($n_1=10, n_2=18$) in a design in which there is a between-subject factor (Groups: 2 levels) and a within-subjects factor (...
3
votes
0answers
408 views

Problem with classifier after using SMOTE to balance the data

We've ran into a problem while training a classifier on an unbalanced data set. The response is binary with 0 indicating 'non defaulter' and 1 indicating 'defaulter' (it's a credit scoring task). ...
3
votes
0answers
297 views

How to use k nearest neighbours for binary classification with unbalanced classes?

I have relatively large (100k items) dataset which I need to split in two groups. So far I've tried knn and the results are not good mainly because I have disproportion in my training data: 90% of ...
2
votes
3answers
375 views

Is it right to build a logistic model for population with 2% of yes and 98% no population with 800k obs and 200 variables

I have a dataset which has has some 800,000 observations data at member level with some 200 features and it has a response flag of 1/0. The proportion of response 1 flag is 2% of entire member ...
2
votes
2answers
105 views

If one group has 23 participants and the second group has 117 participants in it, can we rely on the result of t-test? [duplicate]

I was evaluating a research paper in which the authors have 23 male participants and 117 female participants. They have applied t-test to calculate Gender differences and have even concluded on ...
2
votes
2answers
712 views

kNN and unbalanced classes

Do you think that unbalanced classes is a big problem for k-nearest neighbor? If so, do you know any smart way to handle this?
2
votes
1answer
321 views

Which metric should I trust to evaluate my predictive model

I am working on predictive model and when I evaluate it, I find good accuracy_score, precision_score, ...
2
votes
2answers
89 views

Collecting training data for document classification with unbalanced classes

I have a document classification problem in which the estimated class proportions in the population are severely unbalanced: the population is ~99% class 0 and ~1% class 1. I am using a logistic ...
2
votes
1answer
801 views

SMOTE throws error for multi class imbalance problem

I am trying to use SMOTE to correct imbalance in my multi-class classification problem. Although SMOTE works perfectly on the iris dataset as per the SMOTE help document, it does not work on a similar ...