Questions tagged [unbalanced-classes]
Data organized into discrete categories or *classes* may present problems for certain analyses if the number of observations ($n$) belonging to each class is not constant across classes. Classes with unequal $n$ are *unbalanced*.
999
questions
2
votes
0
answers
38
views
Heavily unbalanced sampling design - Structure of GLMM (ecology)
I am currently working on a dataset (count data) from a rather heavily unbalanced sampling design.
In particular, I would like to be able to predict the abundance of the studied species according to ...
- 73
0
votes
0
answers
3
views
Centering outcomes using sample versus population means with baseline measures
Suppose longitudinal experimental data where outcome outcome $y$ is measured at baseline, units randomly receive a treatment or control condition, then $y$ is measured again. You fit the below model ...
- 691
1
vote
0
answers
28
views
AUC ROC and accuracy for different datasets of same problem
I have datasets that correspond to different traffic load inputs. I am doing binary classification on them. The proportion of 1s to 0s varies from dataset to dataset. E.g., dataset 1 is imbalanced ...
1
vote
0
answers
21
views
Binary probability models: considering event probability above the prior probability
Consider a model of the probability of a binary, yes/no-type of event. The event is infrequent, say it happens only once every thousand times. In that regard, the prior probability of the event is $0....
- 46.4k
2
votes
3
answers
82
views
What is "information leak from test to train" ? Is stratification by target a leak?
It's common practice to do procedures such as standardization and even missing value imputation (commonly based on some means) after train/test split - otherwise it is treated as information leak from ...
- 21
0
votes
0
answers
12
views
Does a newly constructed ML dataset need to have an official train-dev-test split? Should the test set be intentionally balanced?
I have constructed a novel ML (NLP) dataset for classification and labeled it with three classes. The dataset is rather small with about 700 examples, out of which the classes have about 400, 200, and ...
- 1
1
vote
1
answer
31
views
Is it possible of an imbalanced dataset to benefit from the results of the same dataset after balancing?
I am new to data science & machine learning
I am using Weka platform to work on a classification problem with an imbalanced dataset.
My question is: can I apply a feature selection method to a ...
- 11
2
votes
0
answers
14
views
Question about collapsing variables and oversampling minority classes
i have imbalanced data consisting of nine classes, and i am planning to collapse them into two classes. i performed stratified (proportionate) sampling between test, validation, and training sets ...
- 121
0
votes
0
answers
39
views
What is the correct way to apply a feature selection method to an imbalanced dataset?
I am new to data science & machine learning, so I'll write my question in detail.
I have an imbalanced dataset (binary classification dataset), and I want to apply these methods by using Weka ...
- 11
0
votes
0
answers
22
views
gaussian distribution endurance test [duplicate]
I am conducting an endurance test experiment on spare parts. I put these part under stress for long hours to see when they start failing. I have a total amount of 42 spare parts.
1 part failed at run-...
0
votes
0
answers
9
views
How can we develop good priors for group means when centering with unbalanced classes?
Suppose you have data on $y_i$, the outcomes from an experiment in which units were randomly assigned to be in a treated group ($z=1$) or a control group ($z=0$). Let's further assume each group's ...
- 691
1
vote
0
answers
25
views
Partially nested ANOVA
I have a data set that I would describe as unbalanced or partially nested and I am unclear about how to proceed. My data is nested as follows. The number of samples I have is listed in parentheses.
...
- 13
1
vote
0
answers
90
views
Accounting for overrepresentation of positives in binary classification test set for calculation of precision and recall
I have a binary classification task with highly imbalanced data, since the class to be detected (in the following referred to as the positives) is very rare.
For data limitation reasons my test set ...
- 11
0
votes
1
answer
43
views
Proving a class imbalance IS a problem in Machine Learning [duplicate]
Context: Have been trying to create a prediction model for a 1% outcome variable using Random Forest Machine Learning for a large health survey (entirely multi-level categorical data, yes/no outcome, ~...
- 1
0
votes
0
answers
19
views
Empty class in validation set with stratified k-fold cross-validation
I'm working on a multiple classification machine learning problem. The dataset is highly imbalanced, with the smallest class having only 3 samples. To validate the performance, I want to perform ...
- 1
0
votes
0
answers
23
views
Availability of code for implementing the PN rule for imbalanced data
I have been reviewing the paper PNrule: A new Framework for Learning Classifier Models in Data by Agarwal & Joshi (2000) and the associated technical report. The paper outlines an approach to ...
- 339
0
votes
0
answers
19
views
Should I address the imbalance when using CalibratedClassifierCV?
Im using RandomForestClassifier and XGBClassifier with an imbalanced dataset, 1:2 ratio more or less, 1 being the most prevalent class.
My procedure is the following:
Use StratifiedKFold to get ...
0
votes
0
answers
17
views
imbalance and imputation
It has been answered in https://stats.stackexchange.com/q/380668 that class imbalance should have nothing to do with imputation. I agree with the post, but I am concerned about class imbalance.
I am ...
- 1,315
3
votes
1
answer
418
views
Probability Calibration for Highly Imbalanced Binary Classification
I am working on a binary classification problem on a highly imbalanced dataset (1:100) where model probabilities are important for the use case and need to be well calibrated to best represent true ...
0
votes
0
answers
103
views
Ways to Reduce False Positive or False Negatives in Binary Classification (0,1)
I am working on a task in which I need to classify binary labels 0 and 1 properly (as close to perfection as possible). My final dataset (ready for classification) has input data with 141 features and ...
0
votes
0
answers
14
views
Log-binomial or IPTW for Imbalanced sample
I have a binary outcome of depression and exposure variable of a 4-level categorical variable of diet, diet1 group has 300 participants, diet2 has 40, diet3 has 30 and diet4 has 450. For this ...
1
vote
0
answers
26
views
Better understanding classification with unbalanced test data from a mathematical perspective
Suppose I want to get a model, such as a neural network, to correctly classify pictures of cats and dogs and I know that the test set contains around $1\%$ of cats and $99\%$ of dogs.
My intuition is ...
- 177
0
votes
0
answers
36
views
Unbalanced Mixed Measures ANOVA in R
I have a data set that was testing the effect of an intervention on disordered eating from pre-to-post test. There are 45 subjects. 24 subjects were randomized to condition 1. 21 subjects were ...
- 1
0
votes
0
answers
31
views
Should I mimic real life distribution in my training set for good learning?
I have a database of data that is labelled good. I can make new data myself and I'll label it bad (I know how to transform a <...
1
vote
1
answer
59
views
Estimate recall for extremely rare events
I want to estimate the recall of a binary classifier. I have a dataset of ~1B examples but I don't know the ground truth, the only thing I know is that positives are extremely rare. I can randomly ...
- 13
1
vote
0
answers
23
views
3 classes imbalanced and hierarchical classification
I need to classift a dataset with 3 classes:
A - 85% of the data
B - 10% of the data
C - 5% of the data
Where C is a subset of B.
What should be the best way to approach it?
- 57
9
votes
1
answer
319
views
How does class balancing via reweighting affect logistic regression?
When developing machine learning classifiers, some people upsample or upweight the minority class to achieve a 50-50 balance, claiming that this improves performance. Some statisticians have ...
- 10.5k
1
vote
0
answers
36
views
Optimal metric for training with Class-specific masked input features and imbalanced dataset
I have a classification problem of 8-classes, which are extremely imbalanced. The input dataset consists of sequences, each of length n features, where n = 19. For each of the 8 classes, I have a ...
- 111
0
votes
0
answers
20
views
Training a classifier on a highly unbalanced class distribution?
I'm trying to train a text classification model that can predict label $A$ and $B$ accurately. However, 95% of the text examples in my dataset, which is representative of the kind of data I want my ...
- 23
0
votes
0
answers
19
views
Nested and crossed random effects together?
I have a very unbalanced design, as my data is not from an experiment but a summation of different data bases.
I have several hundred individuals for which I have measurements of a continuous ...
- 69
2
votes
3
answers
103
views
The validation set includes few positive labels
I'm training a classifer on an unbalanced dataset. The test dataset's positive proportion is 0.02%.
For that reason, the validation data set labels proportions are the same. Because the validation set ...
- 301
3
votes
2
answers
106
views
Evaluating the classifier on K validation sets, but training it on a fixed training set, when data is imbalanced
I'm training a binary classifier on imbalanced data (The real/production data has ~%2 of positive labels).
Besides the questionable efficiency of oversampling/undersampling technique, I have a lot of ...
- 301
0
votes
0
answers
14
views
I optimized my K Value in a KNN model and it increased ROC however the model suffered in accuracy for the minority class
So I recently set out to build a KNN model and started with a KNN of 5. I received a ROC of .81 and accuracy for the majority and minority class of around .80. After I optimized K to 43 I assumed it ...
- 1
0
votes
2
answers
63
views
Data is unbalanced, but the train and test set don't represent it [duplicate]
I'm training a classifier with a binary target variable.
My data is unbalanced. The problem is that the training data (split into train, val, and test sets) is more balanced than the real data (the ...
- 301
0
votes
1
answer
37
views
Can an imbalanced data set cause decision tree not to split?
If I have an imbalanced response variable 80% majority, 20% minority, and my decision tree is not finding any splits. Could this potentially be because of the imbalance in my response?
0
votes
0
answers
26
views
Accuracy result for imbalance dataset on Neural Network model
Im currently learning Classification Neural Network and Im working with imbalance dataset. I have 2 model with 2 different result and 4 type of classes.
The 1st model there are no sampling or ...
- 23
0
votes
0
answers
31
views
ROC AUC = 0.69 and PR AUC = 0.007
just a question my data is not balanced and to address the imbalance I am using cost sensitive logistic regression however the results are surprising, ROC AUC = 0.69 and PR AUC = 0.007 does the Pr Auc ...
- 1
1
vote
1
answer
332
views
Logistic regression - Does a decision threshold of 0.5 ever make sense?
Say I fit a logistic classifier on a supervised dataset with binary labels.
If I select a threshold of decision of 0.5, which assumption am I implicitly making?
Is there any situation where 0.5 makes ...
- 1,137
0
votes
0
answers
57
views
TimeSeriesSplit for big unbalanced data set: Training data only contain information about one class
I ran into an issue when doing cross-validation with TimeSeriesSplit on my big unbalanced time-series data set
As my Y is 98.1% composed of the same class, when ...
- 1
10
votes
5
answers
1k
views
Could we explain the disadvantage of imbalanced data mathematically?
Simple setup:
observed response is binary (yes/no, 0/1, positive/negative).
use logistic regression to model the probability of the response being, say, 1: $P(Y=1|X)$.
the MLE of the model ...
- 125
0
votes
0
answers
99
views
What Does AUC for Precision Recall Curve Stand For?
Similarly to What does AUC stand for and what is it?, I'd like to know the interpretation of the AUC for the Precision Recall curve.
One can calculate the precision recall curve:
Then easily ...
- 129
0
votes
0
answers
113
views
α-balanced focal loss - why we actually decrease the importance of positive class
This is the equation for Focal Loss. The loss is an extension of weighted cross entropy, and aims to balance the impact of majority of easy negative class samples. The α parameter is a weighing term ...
- 646
0
votes
0
answers
21
views
Classification email newsletter imbalanced
I'm given a case to determine the best time to send an email newsletter based on whether the email is opened.
The problem is that over 70% of the emails are sent on Tuesday and the dataset is ...
1
vote
0
answers
23
views
why does SVM outperforms KNN in 1-gram but in 2,3,4, and 5 KNN outperforms SVM?
my project is authorship attribution which is a multiclass classification, the number of classes is 150, and the number of documents is 2798; it is also an unbalanced issue some classes have more ...
- 11
0
votes
0
answers
19
views
Why we cannot calculate an ROC curve in cost sensitive learning?
In the Applied Predictive Modeling book, cost sensitivity learning approach, the author(s) write:
One consequence of this approach is that class probabilities cannot be
generated for the model, at ...
2
votes
1
answer
67
views
Calibrarion curve of a logistic Regression model
I have a high imbalanced dataset and I fitted a logistic regression model on it. The calibration curve is:
As you can see there is poor calibration after 50. Is the model bad or the problem is the ...
- 139
2
votes
1
answer
115
views
Poorly calibrated probabilities but good classification in confusion matrix
I have an imbalanced data set. My goal is to balance sensitivity and specificity via the confusion matrix. I used glmnet in r with class weights. The model does well at balancing the sensitivity/...
- 235
2
votes
0
answers
40
views
Resampling to handle class imbalance in logistic regression [duplicate]
I was wondering if anyone could help me understand resampling for class imbalance. From what I have learned, class imbalance is usually a small data problem where the less prevalent class usually ...
- 51
0
votes
0
answers
17
views
Comparing two groups in multi-classification: precision to indicate class imbalance, recall to indicate quality of samples
I have a situation where I need to analyse how a classifier for a multi-classification problem for images performs on two different groups. I need to interpret precision and recall values.
I am ...
- 23
3
votes
0
answers
55
views
loss function for supervised anomaly detection in time series [closed]
I have a supervised anomaly detection problem in a time series data, which the dataset has three columns:
datetime
value(a float number)
label(1 for anomaly, 0 for normal)
It's common that the ...
- 31