Linked Questions
21 questions linked to/from What problem does oversampling, undersampling, and SMOTE solve?
1
vote
0
answers
154
views
Why over/under-sampling could not help my model fitting? [duplicate]
I fit random forest to my imbalanced dataset with minority class 1. I found that the AUC under the imbalanced data was better than that of re-sampled dataset (over/under sampling). Can someone help to ...
- 453
101
votes
8
answers
35k
views
When is unbalanced data really a problem in Machine Learning?
We already had multiple questions about unbalanced data when using logistic regression, SVM, decision trees, bagging and a number of other similar questions, what makes it a very popular topic! ...
Tim♦
- 130k
64
votes
7
answers
49k
views
Binary classification with strongly unbalanced classes
I have a data set in the form of (features, binary output 0 or 1), but 1 happens pretty rarely, so just by always predicting 0, I get accuracy between 70% and 90% (depending on the particular data I ...
- 842
105
votes
4
answers
13k
views
Are unbalanced datasets problematic, and (how) does oversampling (purport to) help?
TL;DR
See title.
Motivation
I am hoping for a canonical answer along the lines of "(1) No, (2) Not applicable, because (1)", which we can use to close many wrong questions about unbalanced ...
- 111k
7
votes
2
answers
6k
views
Removing duplicates before train test split
Let's say you have a dataset generated from real world sampling which has lots of duplicates (the dependent and independent variables are identical) and you want to train a classifier to predict the ...
- 71
8
votes
1
answer
10k
views
random forest for imbalanced data?
I have a dataset where yes=77 and no=16000, a highly imbalanced dataset. My plan was to identify the most important variables influencing the response variable using random forest and then develop a ...
- 139
8
votes
1
answer
2k
views
When is oversampling poor practice?
For my particular domain and problem, I have data on the entire population. However, my "event" only occurs in 0.5% of the cases. I want my model to be able to pick up on significant characteristics ...
- 644
3
votes
1
answer
2k
views
Imbalanced Test Data
I have an imbalanced (1:5) training and test set with only two classes and have oversampled the training set with SMOTE so that the class ratio is 1:1. The ML model gives values over 0.7 for accuracy, ...
- 504
0
votes
1
answer
2k
views
How do I perform a logistic regression w/ SMOTE
I want to understand which variables lead to an infection by parasites in a tree. Hence, I want to use stepwise logistic regression based on AIC. First, I describe what I would do, and then my code ...
- 448
3
votes
0
answers
2k
views
Effects of class imbalance on nn batch training
Say I have a binary classification task, where the positive class (1) is only 1% of the whole data set.
Intuitively I can understand why this could be bad for the classifier as the model may learn ...
- 377
1
vote
0
answers
1k
views
Why would random forest perform bad on unbalanced class
There is a huge number of posts saying that an imbalanced classes are bad. And only half explains it in terms of recall-presicion scores, meaning that accuracy can be high but F1 score low.
What I ...
- 1,135
1
vote
1
answer
701
views
Prediction for imbalanced and small sample sized data
I have to create a classification model where my dataset contains 697 observations which only 18 are from the group of interest. As usual, I split data the into a training and test set stratified by ...
- 111
-1
votes
1
answer
639
views
Imbalance Classification : How SMOTE handles majority class?
I'm working on Imbalance Classification problem with minority class(0.017%).
I've read that imbalance classification can be handled using Undersampling, Oversampling and SMOTE.
Major drawback of ...
1
vote
1
answer
640
views
What are other ways of doing oversampling apart from SMOTE?
I have just begun learning about machine learning techniques and started solving problems on kaggle. I have a few questions about how to handle class imbalance:
How to handle imbalance dataset ...
- 19
1
vote
0
answers
661
views
Neural network for imbalanced data
I have an imbalanced data (n = 600, about 97% majority and 3% minority) with 20 features and a binary outcome. The data has been split into a training set and a test set (80%/20%). I used H2o autoML ...
- 63