Questions tagged [unbalanced-classes]

Data organized into discrete categories or *classes* may present problems for certain analyses if the number of observations ($n$) belonging to each class is not constant across classes. Classes with unequal $n$ are *unbalanced*.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Expert-model and machine learning hybrid approach

Is there a recommended approach how the results of an rule-based expert system can be combined with an ML model? Let's say you have a text classification problem with 100 classes. For some classes you ...
user avatar
  • 1,119
1 vote
0 answers
18 views
+100

Optimize classification rule in multinomial logistic regression

We know that in the case of logistic regression, a classification threshold p=0.5 is generally not an optimal choice when seeking to optimise sensitivity and sensitivity. This is generally due to the ...
user avatar
  • 38
0 votes
0 answers
19 views

Balancing a dataset in object detection through data augmentation and then random oclusion of classes?

first of all some info about the model I'm using. I'm using YOLOR, which uses a YOLOV5 formatted dataset. I'm trying to detect components off of PCB's which are divided into some groups (visible on ...
user avatar
1 vote
0 answers
11 views

How do I perform a train-validation split on data with class imbalance such that the class imbalance ratio is preserved?

My data has class imbalance-- that is, some classes have significantly fewer training samples than the others. I want to perform a train-validation split in such as way that the class ratios are ...
user avatar
2 votes
2 answers
303 views

class weighted classification

I am working on my multi-class classification project and I have a question: I have three classes in proportion: 50%, 47% and 3%. I decided to use class_weight="balanced" parameter in random ...
user avatar
  • 31
0 votes
1 answer
20 views

What metrics work well in unbalanced assemblies?

I wanted to know if there are some metrics that work well when working with an unbalanced dataset. I know that accuracy is a very bad metric when evaluating a classifier when the data is unbalanced ...
user avatar
0 votes
1 answer
33 views

References: t-test and Chi-squared test can be conducted with unequal sample sizes

I have two independent groups: sample size of group 1 is 24000, and sample size of group 2 is 246. With the drastically different sample sizes in the two group, I need to use the Student’s t-test or ...
user avatar
  • 1
0 votes
3 answers
41 views

How to correct Chi-square's p-value when working with very unbalanced contingency tables?

I'm studying the association between a rare disease and smoking. Because the disease is rare, my contingency table is highly unbalanced with way more Non-Diseased than Diseased individuals, ...
user avatar
  • 31
1 vote
0 answers
22 views

Biased logistic regression in pytorch

My model has decently high AUC=90%, but is biased, underestimating the probability $y=1$. This is systematic across some of the input features as well. How can I nudge the bias term, or otherwise ...
user avatar
1 vote
0 answers
17 views

Metrics for imbalanced multi-class classification [duplicate]

I am looking for informations about metrics for classification with 3 unbalanced classes. I have following numbers of samples in every class: 1 As you can see two classes are quite balanced and one is ...
user avatar
  • 31
1 vote
1 answer
21 views

Dependent variable has no variance error in logit regression

I m running a logit regression with over 90,000 observations. However the case when dependent variable =1 , is only 115 observations as per the data, the rest are 0. The Eviews software shows "...
user avatar
  • 31
1 vote
0 answers
24 views

What are some algorithms that are immune to class imbalance, and what makes them so?

This question is closely similar, however, the answer only speaks about Logistic Regression being one example. I am interested in knowing if there are more algorithms that are not affected (at least ...
user avatar
1 vote
0 answers
22 views

Can I use the AUR under the ROC on unbalanced test data?

I have split my data into training and test data, built several prediction models and now I want to evaluate the models using the test data set.The data is very unbalanced so I balanced the training ...
user avatar
  • 11
0 votes
0 answers
34 views

class_weight='balanced' vs high f_beta score for imbalanced logistic regression in sklearn. Please help explain the difference

I have an imbalanced binary classification problem I am trying to solve with the LogisticRegression algorithm in sklearn. As the data is highly imbalanced I am looking at ways to treat the imbalance ...
user avatar
0 votes
0 answers
3 views

Deep Learning Image Detection - Help needed deciphering machine learning loss and accuracy graph and finding solutions to fix model

I have an imbalanced dataset from Google OpenImages of 6 classes Train (starfish=439; Dolphin = 890; Turtle = 1362; Fish = 6216; Jellyfish = 733; Shellfish = 1141 ) Validation (starfish=20; Dolphin = ...
user avatar
0 votes
1 answer
46 views

Do we need to split the data for Unsupervised Anomaly Detection?

I'm struggling with understanding the concept of splitting data for unsupervised anomaly/outlier detection. You can find all approaches here. I found some papers and implementations that didn't split ...
user avatar
  • 385
0 votes
0 answers
27 views

Machine learning, advice on dealing with small datasets + imbalanced classes

I'm hitting a wall on a project I'm working on with the goal of predicting the probability of success for a particular set of data. Right now I'm using Logistic Regression and I'm finding that my ...
user avatar
  • 55
1 vote
1 answer
16 views

Threshold optimization with cross validation

I have an imbalanced dataset; 95% negative class and 5% positive class. I split my data into train (80%) and test (20%) sets. I am using 5-fold cross-validation on the train set to determine the ...
user avatar
0 votes
1 answer
31 views

Selection of informative examples of majority class for undersample using SVM

I have this idea in mind, but I am not sure how to implement it. Suppose I have an imbalanced data that I want to down-sample instances of majority class, such that it becomes equal in size to the ...
user avatar
  • 231
0 votes
1 answer
33 views

Should I MAKE the training set imbalanced?

We are taking anonymised demographic data of n people with a rare medical condition, and trying to train a binary classifier (currently using xgboost) to reach more similar people with let's say a ...
user avatar
0 votes
1 answer
61 views

SMOTE vs Stratified Sampling in highly imbalanced dataset - classification

I am working on a project with the goal of predicting Cerebral strokes from brain arteries data (speed of blood, resistance etc. of one artery and of the neighboring ones). I have a dataset with ...
user avatar
  • 11
0 votes
1 answer
84 views

Upweight minority class vs. downsample+upweight majority class?

I've been getting some conflicting advice from various ML podcasts/videos/articles lately for how to deal with imbalanced datasets. Let's say my independent variable for a classification problem has a ...
user avatar
  • 11
0 votes
0 answers
12 views

Different F-1 scores on dev and test set, more data doesn’t increase performance?

I am fine-tuning a BERT model on sentiment analysis for tweets in a particular language (this model was pre-trained on tweets too). Initially I had an imbalanced dataset of around 9k examples (80% ...
user avatar
  • 181
0 votes
0 answers
52 views

Poor out of time performance

I am working on a behavioral model which predicts the probability of default (PD) during the next 12 months for an existing customer with an outstanding loan. My dataset consists of monthly snapshots ...
user avatar
  • 11
1 vote
0 answers
31 views

Dealing with class imbalance and data complexity issues

I am doing a classification task for a 5-class imbalanced dataset. Class distribution shows 2-majority & 2-minority clasess, as far: ...
user avatar
  • 231
2 votes
2 answers
147 views

Can NaN class be assigned to a certain class in imbalanced datasets (binary classification)?

I'm working on a spam detection binary classification problem, but the dataset is very imbalanced (99% to 1%). I know there are techniques like over/under sampling, but I don't think it can be used in ...
user avatar
  • 41
0 votes
1 answer
31 views

Unbalanced groups in repeated measures anova

I am analysing metabolomics data (~1,000 measurements) across 30 samples from 10 subjects. The samples are from 2 treatments groups of 5 subjects from whom 3 muscle samples were taken from 3 locations ...
user avatar
  • 1
0 votes
0 answers
40 views

Oversampling in Longitudinal/Panel Data

Would make sense to apply any oversampling (e.g. SMOTE et similia) techniques in order to balance the outcome classes in the context of longitudinal/panel data? Wouldn't such procedures ignore the ...
user avatar
0 votes
1 answer
36 views

Can I use lmm (lmer) for an unbalanced design?

I'm trying to analyze a data set but I'm having trouble finding the right analysis. I'm looking at performance (Work) of a material. Unfortunately my experimental design turned out to be unbalanced. I ...
user avatar
0 votes
0 answers
29 views

Mixed effects model or mixed factorial ANOVA? Unequal sample sizes

I am trying to figure out which analysis to run for an experiment. My independent variables are self-reported learning styles (dichotomous, between-subjects), stimulus type (dichotomous, within-...
user avatar
  • 1
1 vote
1 answer
73 views

What does high auc score but poor f1 indicate for imbalanced dataset?

I am working on a binary classification with an imbalanced dataset of 977 records (77:23 class ratio). My label 1 (POS not met) is the minority class. Currently without any over/under sampling ...
user avatar
  • 1,758
0 votes
0 answers
32 views

Best Eval Metric for Credit Scoring: ROC AUC/PR AUC/F1?

My team is developing a credit scoring model for a situation in which... The positive class accounts for 10% of the training data FNs (predicting no default for actual default) costs us ~\$10-15K FPs ...
user avatar
5 votes
2 answers
466 views

"Dumb" log-loss for a binary classifier

I am trying to understand how I can best compare a classifier that I have trained and tuned against a "dumb" classifier, particularly in the context of binary classification with imbalanced ...
user avatar
1 vote
1 answer
79 views

Balanced accuracy reduces to accuracy for balanced datasets

This question might be trivial, but I have problems understanding this line taken from here: The balanced_accuracy_score function computes the balanced accuracy, which avoids inflated performance ...
user avatar
  • 113
0 votes
0 answers
42 views

What's the difference between combined pos_bagging_fraction and neg_bagging_fraction vs is_unbalance vs scale_pos_weight in LightGBM?

Let's suppose a binary classification task and an unbalanced dataset (10% of positive records). I am using LightGBM and would like to better understand the difference between the combined ...
user avatar
  • 33
1 vote
1 answer
31 views

How can sampling methods change the probabilities in imbalanced classification?

I just wondered how sampling methods work in imbalanced cases. So, I used imbalanced dataset(almost, 99:1) and logistic regression for Binary Classification. And the results are as below. Now, I have ...
user avatar
0 votes
0 answers
9 views

How to achieve good result in text classification when the data is very small?

I have a dataset with many user comments, I want to classify this dataset with label 0 or 1. The dataset only has 730 comments labeled as 0 and 65 labeled as 1. I have developed a simples model using ...
user avatar
0 votes
1 answer
38 views

imbalanced numeric target variable in machine learning

In the machine learning problem I'm trying to solve, the target variable is numeric.(Integers 0, 1, 2, 3, ... 25. However, they are highly imbalanced. There are more than 20k of 0s, 1500 of 1s, 600 ...
user avatar
  • 659
0 votes
0 answers
16 views

Does the imbalance in my data affect model quality in this case?

I have two datasets, used for classification task: training and validation one. Both of them are equally imbalanced, with about 24% of target value equal 1 and 75% of target value equal 0. I am using ...
user avatar
  • 113
0 votes
0 answers
11 views

How can I make a text classification model that can understand the meaning of a word?

I am new to NLP and recently I have been working in a text classification model with 684 texts classified as 0 and only 77 classified as 1. My result so far is not bad, 74% precision and 77% recall. ...
user avatar
0 votes
1 answer
64 views

Does it matter if real data will be imbalanced, if the ML model was trained on a balanced dataset?

I have trained a machine learning model (supervised, classification, LinearSVC) on a balanced dataset, which produces relatively good results on the test data. I am happy with the numbers, but not ...
user avatar
0 votes
0 answers
9 views

Unbalanced and missing data in generalized mixed effects model

I have a highly unbalanced dataset from observational data. I want to assess sources of variation on my dependent variable (Y, binomial distribution), controlling by the year of sampling. For that, I ...
user avatar
  • 1
2 votes
3 answers
100 views

Effects of class imbalance on neural network weights

My question is about unbalanced classes problem in case of a classifier neural network for natural language processing (in particular, a neural network with LSTM). I want to train a neural network to ...
user avatar
1 vote
0 answers
69 views

PRAUC curve Interpretation on imbalanced data

My training data is undersampled to a positive to negative ratio of 5%. I observe a PRAUC of .4 on my training data. When I test the model on real-world data where the positive to negative ratio is .5%...
user avatar
  • 21
-1 votes
1 answer
115 views

How to use SMOTE on the final model training?

I have three datasets: train, validation, and test (all datasets are labeled). When I have tuned the hyperparameters using random search, I applied SMOTE just on the train data. Now, after I found ...
user avatar
  • 67
7 votes
0 answers
127 views

Are there Imbalanced learning problems where re-balancing/re-weighting demonstrably improves *accuracy*?

I have been looking into the imbalanced learning problem, where a classifier is often expected to be unduly biased in favour of the majority class. However, I am having difficulties identifying ...
user avatar
0 votes
0 answers
10 views

How do you perform a mixed model ANOVA with nested and crossed factors?

I have a dataset where measurements were made on individuals from three different genotypes at two different temperatures. Thus, I have two crossed factors- Temperature and Genotype. However, separate ...
user avatar
4 votes
1 answer
726 views

Why is it that if you undersample or oversample you have to calibrate your output probabilities?

There are a number of stackexchange posts saying that you have to calibrate your probabilities if you oversample and undersample like 1,2. But my question is why? Lets use an example of detecting spam ...
user avatar
  • 157
0 votes
1 answer
72 views

PCA with unbalanced panel

As far as I know, to use Principal Component Analysis (PCA) on a panel of data, data must be balanced. As an example, consider the returns of the constituents of S&P500 from 1967 to 2020. Because ...
user avatar
  • 36
2 votes
1 answer
118 views

Does assigning class weights allow for the use of accuracy metrics that require balanced datasets?

If you have an unbalanced dataset, but you assign (inverse) class weights when fitting, does this mean that model loss and accuracy metrics will be computed to allow for using ROC AUC and accuracy, ...
user avatar

1
2 3 4 5
19