Questions tagged [unbalanced-classes]

Data organized into discrete categories or *classes* may present problems for certain analyses if the number of observations ($n$) belonging to each class is not constant across classes. Classes with unequal $n$ are *unbalanced*.

Filter by
Sorted by
Tagged with
9 votes
5 answers
1k views

Could we explain the disadvantage of imbalanced data mathematically?

Simple setup: observed response is binary (yes/no, 0/1, positive/negative). use logistic regression to model the probability of the response being, say, 1: $P(Y=1|X)$. the MLE of the model ...
  • 113
0 votes
0 answers
46 views

What Does AUC for Precision Recall Curve Stand For?

Similarly to What does AUC stand for and what is it?, I'd like to know the interpretation of the AUC for the Precision Recall curve. One can calculate the precision recall curve: Then easily ...
0 votes
0 answers
22 views

α-balanced focal loss - why we actually decrease the importance of positive class

This is the equation for Focal Loss. The loss is an extension of weighted cross entropy, and aims to balance the impact of majority of easy negative class samples. The α parameter is a weighing term ...
  • 626
0 votes
0 answers
16 views

Classification email newsletter imbalanced

I'm given a case to determine the best time to send an email newsletter based on whether the email is opened. The problem is that over 70% of the emails are sent on Tuesday and the dataset is ...
1 vote
0 answers
20 views

why does SVM outperforms KNN in 1-gram but in 2,3,4, and 5 KNN outperforms SVM?

my project is authorship attribution which is a multiclass classification, the number of classes is 150, and the number of documents is 2798; it is also an unbalanced issue some classes have more ...
0 votes
0 answers
13 views

Why we cannot calculate an ROC curve in cost sensitive learning?

In the Applied Predictive Modeling book, cost sensitivity learning approach, the author(s) write: One consequence of this approach is that class probabilities cannot be generated for the model, at ...
2 votes
1 answer
45 views

Calibrarion curve of a logistic Regression model

I have a high imbalanced dataset and I fitted a logistic regression model on it. The calibration curve is: As you can see there is poor calibration after 50. Is the model bad or the problem is the ...
  • 79
1 vote
0 answers
21 views

Poorly calibrated probabilities but good classification in confusion matrix

I have an imbalanced data set. My goal is to balance sensitivity and specificity via the confusion matrix. I used glmnet in r with class weights. The model does well at balancing the sensitivity/...
  • 225
2 votes
0 answers
33 views

Resampling to handle class imbalance in logistic regression [duplicate]

I was wondering if anyone could help me understand resampling for class imbalance. From what I have learned, class imbalance is usually a small data problem where the less prevalent class usually ...
0 votes
0 answers
13 views

Comparing two groups in multi-classification: precision to indicate class imbalance, recall to indicate quality of samples

I have a situation where I need to analyse how a classifier for a multi-classification problem for images performs on two different groups. I need to interpret precision and recall values. I am ...
3 votes
0 answers
32 views

loss function for supervised anomaly detection in time series [closed]

I have a supervised anomaly detection problem in a time series data, which the dataset has three columns: datetime value(a float number) label(1 for anomaly, 0 for normal) It's common that the ...
2 votes
0 answers
14 views

Random performance of AUPRC

I've been trying to understand how to interpret what random performance would be for a model I have on the AUPRC score. By 'random performance' I mean the worst possible performance. Purely ...
  • 141
0 votes
0 answers
10 views

What is the proportional impact of different group sizes on a linear mixed model?

I am working with a linear mixed model of species. My model has a random slope, and a fixed intercept, with the random effect being based on taxonomic group. I have 4000 data points in my dataset, and ...
0 votes
0 answers
14 views

MCMC model: how to study the influence of personality (repeated measures) on a trait

I am trying to investigate how frogs level of aggressiveness influence their plastic responses to a more or less big opponent. To do so, I presented an individual with a simulated opponent, ...
1 vote
1 answer
67 views

XGBoost poor calibration for binary classification on a dataset with high class imbalance

I've read a lot of threads/questions about this issue and I got conflicting answers. I've trained an XGBoost model on tabular data to predict the risk for a specific event (ie a binary classifier). ...
0 votes
0 answers
45 views

The alpha parameter of focal loss

I want to use a weighted focal loss for my imbalanced object detection problem. \begin{equation} L = - \alpha(1-\hat{p})^\gamma log(\hat{p}) , \ \hat{p} = \begin{cases} p, & \text{if}\ ...
0 votes
1 answer
73 views

Balancing dataset for survival analysis

So I'm using survival analysis to help identify customers at risk of churning. As of now, I've been using a hugely imbalanced dataset that consists of 99.5% of customers in the majority class (i.e. ...
2 votes
2 answers
114 views

Best way to obtain probabilities and model explanations with imbalanced data

I am currently working on machine learning problem with the following characteristics:  - Data have binary outcomes and are severely imbalanced (positive class is ~0.5% of my sample of ~500,000 data ...
0 votes
1 answer
107 views

Multiclass Unbalanced Classication :Very very low F1 scores, high precision low recalls

have three classes for sentiment (negative, neutral, and positive). I created synthetic fake data for the positive class the analogy now is 50% neutral, 45% positive, 5% negative. I get the metrics ...
2 votes
1 answer
47 views

Why although my precision for class 1 is very low (0.05), and the recall is 0.92, the Precision Recall plot shows decent results?

I have trained a classifier I am experimenting on, using a highly imbalanced data set (284,807 transactions and out of them 370 are of class 1) and I get the following results. ...
1 vote
0 answers
33 views

Statistical significance of repeated cross validation

I'm building a risk model for predicting risk of event within a specific time horizon for some patients. I use ROC-AUC score to evaluate the model. My dataset is highly imbalanced (15 events within ...
4 votes
1 answer
151 views

Training with extremely imbalanced Dataset

I have a object detection problem which has extremely imbalanced dataset. Lets say there is only one class to detect, say banana or not banana. This detection network will be used in a real case where ...
  • 41
1 vote
1 answer
28 views

Poor model training- what next?

I am trying to train models to predict if somebody will get breast cancer- it is a binary classification problem, using limited features that replicate data a patient's primary care physician will ...
0 votes
0 answers
12 views

Validation loss starting good then follow the training loss [duplicate]

I have faced this problem and I don't know if it's something that occurs as usual or something that I need to fix. I have an unbalanced dataset 90:10; performing the weight initialization my ...
7 votes
4 answers
499 views

Data Imbalance: what would be an ideal number(ratio) of newly added class's data?

Assume that I have 10 classes with 100 samples for each class—same # of samples, perfect balanced dataset. I want to add 3 new classes, and which of the following is the best option for the number of ...
0 votes
0 answers
30 views

Transfer learning with logistic regression and additional features

I have a logistic regression model's weights developed from another dataset. I also have a dataset with much fewer sample for one class, so the performance of the models developed on this dataset is ...
  • 135
0 votes
0 answers
28 views

Effectively evaluate a model with highly imbalanced and limited dataset

(This question was originally posted on the Data Science stack.) Motivation Most data imbalance questions on this stack have been asking How to learn a better model, but I tend to think one other ...
2 votes
1 answer
46 views

Control group much much larger than test group

I'm a complete noobie in statistics and this is just me being curious about a study I read online. In the study they had 150 people who had tested positive for disease X, and a control group of 4000 ...
  • 21
0 votes
1 answer
33 views

In which specific situations is minority class oversampling useful? [duplicate]

I understand that, in the context of a binary classification problem, downsampling the majority class is a useful strategy to come up with a smaller, computationally friendly dataset. Using this ...
2 votes
1 answer
29 views

Is it possible to learn/separate this data?

I have a highly imbalanced (~0.4% minority class) binary classification dataset of time series (flux) observations, and am now at a loss on how to classify it, as the data aren't separable. Here's ...
0 votes
1 answer
96 views

Choosing class-balance of training dataset for unbalanced binary classification problem

There are many discussions on here about techniques for handling unbalanced datasets, eg. Are unbalanced datasets problematic, and (how) does oversampling (purport to) help?. My question is different ...
  • 1,723
1 vote
0 answers
32 views

Data Imbalance in Contextual Bandit with Thompson Sampling

I'm working with the Online Logistic Regression Algorithm (Algorithm 3) of Chapelle and Li in their paper, "An Empirical Evaluation of Thompson Sampling" (https://papers.nips.cc/paper/2011/...
  • 11
0 votes
0 answers
22 views

K-fold Stratified cross validation on a dataset with examples of variable length

I have a dataset of audio recording of variable length with large std which are heavily imbalanced in terms of total duration per class. So in stratified k-fold cross validation I would like to main ...
  • 11
1 vote
0 answers
22 views

Reliably evaluate model performance with very few positive samples

I do a binary classification in the domain of predictive maintenance. Setup My dataset is highly imbalanced with only 17 samples of the positive class, but an nearly indefinite amount of negative ...
  • 525
0 votes
0 answers
49 views

Imbalanced classification with xgboost in python with scale_pos_weight not working properly

I am using xgboost with python in order to perform a binary classification in which the class 0 appears roughly 9 times more frequently than class 1. I am of course using ...
  • 171
0 votes
0 answers
18 views

How to deal with training and test data that have different imbalances?

I have a dataset made of categorical variables and a binary outcome. Responses about the variables and the outcome were recorded at time 1 and time 2. The data is imbalanced between the two outcomes (...
0 votes
0 answers
36 views

How to do pruning and set class weights using RPART on unbalanced data?

I'm trying to work on this heart disease dataset by doing binary classification using RPART trees on data that has a hard unbalance, only 8% of the instances are positives. When it comes to pruning I ...
  • 1
1 vote
0 answers
34 views

At what point in the ML pipeline should I under/over sample?

I have an imbalanced multi-class dataset, and am under/over sampling to balance it out. My questions is when should I do this resampling? Should it occur before creating the test set, before creating ...
  • 163
1 vote
1 answer
64 views

Dealing with very small and unbalanced data

I am working on some TV series data, so the number of records is very limited. I have 58 instances, one for each existing episode, which I have randomly split in 45 and 13. The main goal is to make a ...
  • 109
0 votes
0 answers
23 views

Imbalanced classification model training

Consider a dataset with 11 variables where 1 variable Y is considered the label having binary values of 0 and 1. Consider also that in a dataset like this the 0s are far more than the 1s( i.e. 9000 ...
2 votes
2 answers
92 views

few images in validation and test set

I've a dataset with about 123 images (two categories, 19 defect and 104 no defect). I've to implement a classifier so I've decided to split my data in train (70% of all data), validation (20% of all ...
3 votes
2 answers
43 views

Performance evaluation with non-representative data

I am currently trying to apply some models for text classification on a binary task. The core two approaches that I follow are, on the one hand, using word2vec vector representations on a Random ...
  • 43
0 votes
0 answers
48 views

How to handle imbalance with neural networks?

I am trying to do classification on multi-class and multi-label data using feed-forward networks. I have mostly been using keras. Here's all that I have tried to address imbalance in my text data, but ...
0 votes
1 answer
78 views

Perform Rose Method, then Logistic Regression and do k -fold cross validation

I have unbalanced data so I want to oversample obs from the minority class and then apply Logistic regression to the training set. After that, I would like to perform cross-validation. My question is: ...
  • 79
2 votes
1 answer
43 views

How to interpret a low level of classification?

There is a marked dataset of $n=2879$ objects on 7 classes. Each object (an object is a text of 1-3 sentences) was marked up by different users. Class 1 includes $n_1=790$ objects, Class 3 has the ...
  • 850
0 votes
0 answers
25 views

Interpret precision and recall curve

I am evaluating a classification model and I am using the PR curve because of my highly imbalanced dataset (negative class is the 5%). In the end I'm comparing the training PR curve and the test PR ...
1 vote
0 answers
162 views

Neural network for imbalanced data

I have an imbalanced data (n = 600, about 97% majority and 3% minority) with 20 features and a binary outcome. The data has been split into a training set and a test set (80%/20%). I used H2o autoML ...
1 vote
1 answer
35 views

Image classification using Binary Cross Entropy but with only training examples for one of the classes e.g. class 1 VS anything else

I am training a 'specialist network' to reconstruct images of an object using a Variational Autoencoder (VAE). The training set (~15000 images) is of a single object in multiple poses. I also want ...
  • 23
0 votes
0 answers
106 views

Using the Dunn test to get the CLD grouping for the Skillings-Mack test

I have a set of phenotypic data collected from 9 different assays of a set of different plants. Each plant was assayed at least one time, but each individual assay had a different set of plants. So ...
2 votes
1 answer
151 views

Match number of positives in unbalanced data set

I am dealing with a very unbalanced binary classification problem: 1% positives, 99% negatives. Training set is around 10 million rows, 40 columns. I choose the decision threshold (cutoff) on the ...

1
2 3 4 5
20