Data organized into discrete categories or *classes* may present problems for certain analyses if the number of observations ($n$) belonging to each class is not constant across classes. Classes with unequal $n$ are *unbalanced*.

learn more… | top users | synonyms

3
votes
0answers
68 views

Precision in unbalanced multi-class problem

I am dealing with a multi-class classification problem and I compute micro-averaged evaluation metrics (precision, recall and F-measure) by performing 10-fold cross validation. However, the fact that ...
2
votes
0answers
164 views

Is using Rpart with unbalanced data a good idea?

I have a rather unbalanced data set and want to use rpart to build a classification tree. After building the full tree, I prune it back using the 1-SE rule. On average, only 1-2 splits are suggested. ...
1
vote
1answer
71 views

Is it valid to get better performance in logistic regression using only a subset of the coefficients?

I have an imbalanced data set containing 12% of the positive class 88% negative. First, I ran a logistic regression with all my coefficients and got an average accuracy of 0.91 (I know that's not ...
2
votes
1answer
142 views

Which cost function out of Logloss, AUC & overall error is better for unbalanced classes & why?

Why does Logloss & AUC perform better than overall error for unbalanced classes? How to choose between Logloss & AUC or unbalanced classes? FYI - I am referring to objective / cost function ...
1
vote
0answers
42 views

Class imbalance problem and baseline classifier

I have a dataset with four numerical attributes and a class (target) variable. There is an enormous imbalance between positive and negative instances according to class variable. To cope with ...
0
votes
1answer
34 views

Subset of training set produces good results while full training set produces poor results

I have an extremely unbalanced data set: around 200 positive samples and 70,000 negative samples. To overcome this problem I have tried to over-sample the minority class as suggested in previous ...
0
votes
1answer
23 views

How do you evaluate the performance of a classifier if its F1 is higher for one class but low for another?

For a binary classifier, how do I evaluate the performance if I'm getting very high precision & recall values (~0.9) for one class, say A, but lower (~0.5-0.6) values for the other class, say B? ...
0
votes
1answer
133 views

How to train classifier for unbalanced class distributions?

I attempted a ReLU neural network to classify data sets of 3 classes that are not balanced (in both training and test sets), i.e. 30% of samples are in class A, 10% in class B and 60% in class C. And ...
2
votes
2answers
374 views

Estimating classification probability, with low event rates — options other than logistic regression?

I am trying to predict the probability of occurrence of a low event rate outcome (~2% readmission risk after hospital discharge in the population of interest). With the available limited predictors, ...
0
votes
0answers
27 views

Why do I get nonmonotonic performance of linear SVM as I change binary class weight?

I have an unbalanced binary text classification task that I am trying to solve using Liblinear's [L2R_L2LOSS_SVC_DUAL][1] ...
0
votes
1answer
84 views

How to deal with unbalanced data and large dataset on low budget?

If we have a dataset with 5:1 Ratio and 500.000 observations we can randomly sample the majority class getting in this case 100.0000 minority class and 100.000 majority class? I'm wondering this ...
0
votes
0answers
102 views

Creating folds in cross validation

I have a question regarding cross validation. I have training data with response variables. Right now my code to split the data is: ...
0
votes
0answers
14 views

Implications of sampling complete data set prior to classification

I want to ask about the consequences or things to keep in mind when sampling the complete data set to overcome class-imbalance issue. I have come across numerous examples where only the training set ...
1
vote
1answer
145 views

Multi-class classification with imbalanced classes

I have a data from 5 classes and I would like to build a classifier. However the number of feature vectors in each class is very different. One has about 5000, one about 200,000, one about 1,000,000,...
0
votes
0answers
34 views

Orthogonal contrasts in a logistic mixed-effects model with unbalanced dataset

I have a dataset containing a dichotomous outcome variable Y, and 3 independent variables: ...
0
votes
1answer
454 views

R - Classification ctree {party} - Testing sample and leaf attribution with unbalanced data

Let's start with data description of the website visits I analyse : 6M rows Dependant variable quotation is binary and takes values ...
0
votes
0answers
39 views

Class assignment across multiple categories

I currently have a dataset with four segments that were created off of survey data that we are trying to score records into one of the four segments based on another set of behavioral data. The ...
1
vote
0answers
73 views

For classification w unbalanced datasets, is class-weighing the same as oversampling?

in unbalanced classification problems, I find myself using class_weigh = "auto" or similar parameters often, but I don't think I'm fully understanding what it's doing. I know that it's the industry ...
2
votes
3answers
408 views

Is it right to build a logistic model for population with 2% of yes and 98% no population with 800k obs and 200 variables

I have a dataset which has has some 800,000 observations data at member level with some 200 features and it has a response flag of 1/0. The proportion of response 1 flag is 2% of entire member ...
0
votes
0answers
51 views

Need advice on unbalanced time-series dataset, for use with CAPM regression

I have 40 years of monthly historical returns of around 3000 mutual funds. The dataset contains both active and inactive funds, so some funds have data for the whole period, whereas others will have ...
3
votes
1answer
225 views

What's the measure to assess the binary classification accuracy for imbalanced data?

Now I have binary classification problem with positive samples roughly 100 times the number of negative samples. In this case the normal accuracy measure (predict == label) is not a good measure. What ...
3
votes
1answer
733 views

Class weights in caret

I'm using the R package caret to generate classifiers using a variety of different models on an imbalanced dataset. To overcome the class imbalance problem, I am using the "weights" parameter in the "...
0
votes
0answers
105 views

softmax in nnet with cost function

I have 40 classes in a classification problem. I'm using nnet with softmax but since the classes are very imbalanced I get the same probabilities for every case to predict. I read about F1 score. Is ...
0
votes
0answers
17 views

(logistic regression with imbalanced data) Does high polynomial degree in combination with rebalancing negatively affect accuracy?

Data Set: https://www.kaggle.com/c/GiveMeSomeCredit/data (cs-training.csv) Training Tool: Weka Data Processing Tool: Python (for higher polynomial degree) Question: Balancing data (by down sampling ...
1
vote
2answers
81 views

How to choose sampling method for imbalanced data?

I have an imbalanced dataset with 4995:5 ratio as well as other datasets with less imbalanced ratios. I split this 4995:5 ratio into training and testing for about 2/3 training and 1/3 testing. I also ...
0
votes
0answers
151 views

Sampling, feature selection and preprocessing in cross validation

To brief my question, I want to clarify the order of parameter tuning and the correctness of the flow in my scheme. In my classification scheme, there are several steps including: SMOTE (Synthetic ...
1
vote
2answers
370 views

what is the difference between area under roc and weighted area under roc?

Thanks in advance for the help. I have an unbalanced dataset that I am using for a binary classification problem. The classes are unbalanced. I believe that in such a case that weighted area under ...
2
votes
3answers
739 views

What balancing method can I apply to a imbalanced data set?

I'm trying to solve one classification problem from the UCI database repository. Unfortunately (or fortunately), I've noticed that my dataset is imbalanced. I've structured the data as 5 classes, ...
1
vote
0answers
26 views

Should the number of normal samples always be more than that of anomalous samples in training set for anomaly detection?

I am trying to train an anomaly detection algorithm (one-class svm) on a data set with a few hundred positive samples and several thousands negative examples. Is it mandatory that I train the model ...
1
vote
2answers
254 views

What is the best measure for unbalanced multi-class classification problem?

What are some possible classification metric for an unbalanced problem ? Due to skeweness of the distribution, accuracy value is not so meaningful. For instance, if I predict all the classes to class ...
3
votes
1answer
1k views

How to reduce number of false positives?

I'm trying to solve task called pedestrian detection and I train binary clasifer on two categories positives - people, negatives - background. I have dataset: number of positives= 3752 number of ...
0
votes
1answer
950 views

SVM parameter tuning for unbalanced classes (with class weights)

I am trying to run an SVM on an imbalanced dataset (0-90%, 1-10%) using the e1071 package, with the radial kernel. I am using cross-validation to select the best gamma and cost. Additionally, I want ...
1
vote
2answers
537 views

Classifer for unbalanced dataset?

Is there any classifer that can natively support unbalanced datasets? Or what best practices you can suggest to handle such datasets? For example I want to solve task called "pedestrian detection" ...
1
vote
2answers
87 views

How is the chance-level confusion matrix calculated?

I applied an ML technique on my dataset, and got this confusion matrix: 0 1 0 162 62 1 27 50 Funnily, the overall accuracy is worse than ...
2
votes
1answer
386 views

What is the chance level accuracy in unbalanced classification problems?

Suppose one has a balanced classification problem (50% of 0's and 50% of 1's). In such a case, the so called chance-level accuracy of classifier would be 50%. What is the chance-level accuracy if the ...
0
votes
1answer
115 views

Help for interpreting SVM cross-validation results

I am using support vector machines for an unbalanced binary problem (0: 25%, 1: 75%). I do K-fold cross-validation with $K=10$. The metrics I get are: 80% classification accuracy on average for the ...
2
votes
1answer
102 views

12 firms and a total of 204 observations, can I use pooled OLS with firm-dummies or should I use fixed factor?

I am studying the effect of government ownership on firm performance, more specifically I am studying the effect of the government reducing their share in companies which are already partly privatized....
1
vote
2answers
107 views

Bias-Variance tradeoff for classifying unbalanced classes

I would like to use Bias-Variance trade-off to evaluate training set size in a classification problem. There are two classes which are not balanced (~70/30) and it seems that the common use of ...
1
vote
1answer
89 views

Skewed Classification Problem

So I've read around and seen this is a problem. I have a classification problem and 12 variables ... I'm working on getting more, but even if l get the number to 20-30 I feel like the problem will ...
0
votes
1answer
85 views

problems in doing logistic regression with unbalanced sample, give me some references

I have a dataset with lots Y=0 and few Y=1. I have to run logistic regression, so I'm using a retrospective sample in order to get a more balanced sample. Could someone give me some references that ...
5
votes
2answers
139 views

Bias Correction for Large Scale Logistic Regression with Rare Events

I have a large dataset constituted of many ad impressions. My dependent binary variable clicked describe whether or not the ad was clicked on. As you can expect, the number of clicks is about 1000x ...
0
votes
1answer
52 views

Extreme unbalanced design: Group A: 43, Group B: 15,000

I've been asked to review a study comparing two groups, in which Group A=43 subjects and Group B=15,000 subjects. Intuitively, I feel that this is not a valid design, but can't find any specific ...
0
votes
1answer
298 views

Choosing a good binary classifier to be trained by a small set of labeled data

I have a small set of labeled data (diagnosis in individual subjects): ~50 of "sick" observations ~100 of "healthy" observations In reality, only ~1% of the observations are expected to be ...
2
votes
1answer
66 views

What does it actually mean for classes to be balanced?

I saw the following statement when reading Kuhn's APM: "The classes are fairly balanced; there are 111 samples in the first class and 97 in the second..." I thought balance would require the ...
3
votes
3answers
363 views

How can I derive confidence intervals from the confusion matrix for a classifier?

I have am using k-fold cross validation to generate a confusion matrix for a classifier. I need to calculate 95% confidence intervals for the number of times each class is predicted when run against a ...
2
votes
0answers
50 views

Unbalanced groups and classification errors

I would like to adopt a general strategy for dealing with an very unbalanced dataset, where my "positive" group corresponds to 1/40 of all the observations. The reason why I ask it is because all the ...
4
votes
2answers
306 views

Finding occurrences of specific patterns in time series

I have to locate occurrences of Cyllinder, Bell and Funnel patterns in univariate time series $X$ of gamma-ray sensoring. This is a specific case of the general CBF synthetic problem found in a few ...
1
vote
1answer
136 views

Selection bias and reliability

I need a bit of help with interpretation of classification results. I have unbalanced data set (80% = 0 20% = 1), fitting classifiers (SVM, GradientBoosting or kNN) on such data does not yield good ...
1
vote
1answer
886 views

ROC curves for unbalanced datasets

Consider an input matrix $X$ and a binary output $y$. A common way to measure the performance of a classifier is to use ROC curves. In a ROC plot the diagonal is the result that would be obtained ...
1
vote
0answers
139 views

Non-parametric Levene's test by Nordstokke and Zumbo

The example they mention is using a one-way ANOVA. What if I have two factors (3x11) and a dependent variable, can I do a two-way ANOVA to calculate the univariate levene's test? If so, how would I ...