Linked Questions
101 questions linked to/from When is unbalanced data really a problem in Machine Learning?
4
votes
4
answers
3k
views
How to handle data imbalance in classification? [duplicate]
I am working on a text classification problem. My data is highly imbalanced. For example, one category has 700 documents while the other has 30. I have around 30 categories. I tried different ...
- 227
8
votes
2
answers
1k
views
Should we balance the data set if the data is intrinsically unbalanced? [duplicate]
Say I want to predict the cancer rate(regression)/predict the whether a person has cancer or not(classification). The data intrinsically has few cancer patients/low cancer rate, say 1/200. And the ...
- 101
1
vote
0
answers
2k
views
Repeating rare examples in unbalanced data classification [duplicate]
So I'm trying to train a neural network for a rare event detection. Based on that, I have like 1000 times more examples for non-target (everything else) examples that I have for target examples. So I ...
- 121
0
votes
2
answers
627
views
Machine Learning with Skewed Classes in R [duplicate]
I am looking for some suggestions on what methods are appropriate for training a dataset with a high skew in the outcome classes. The ratio of Class 0: Class 1 is about 20:1 and I am looking to ...
- 805
0
votes
1
answer
898
views
Does it make sense to up-sample imbalanced data if I care about predicting the right probabilities? [duplicate]
I'm developing a predictive model on a dataset of about 25K observations with the response variable having ~60 classes. There are ~130 predictor variables, all of them binary. In this problem I care ...
- 13
2
votes
1
answer
689
views
When do we speak of imbalanced data? [duplicate]
I was wondering, what the threshold is for a dataset to be called 'imbalanced'?
Technically every dataset where the target-classes aren't evenly distributed is not balanced. Yet small imbalance ...
- 143
2
votes
2
answers
251
views
Does the ratio of training data (significantly) affects the result? [duplicate]
Say I have two training data of email titles with 10000 entries each. One of them have 2000 ham and 8000 spam, while the other have 5000 ham and 5000 spam. Will predicting any random email with the 2:...
1
vote
0
answers
490
views
Classification model on a highly unbalanced dataset [duplicate]
I’m dealing with a highly unbalanced dataset where 20% of data belongs to class A and 80% belongs to class B.
It’s very hard for us to produce synthetic class A data.
Just wondering if the below ...
- 211
5
votes
2
answers
151
views
Problems with classification in imbalanced datasets [duplicate]
I often read about the problematic of doing classification in imbalanced datasets and methods to address it. Namely, off-the-shelf classifiers learn to minimize some form of total miss-clasffication ...
- 171
1
vote
3
answers
272
views
Binary classification with imbalanced classes [duplicate]
I have a manufacture dataset of 65 million rows corresponding to 65 millions distinct items.
Out of those 65 millions, I have 60,000 of them that failed a certain test, thus I have very imbalanced ...
- 129
2
votes
0
answers
245
views
Why imbalance in the data set is an issue in data mining and machine learning? [duplicate]
I've analyzed a data set from a credit card company before and it has the famous "unbalanced classes" problem like all the other credit card companies, i.e., in the data set (the information of users ...
- 6,919
1
vote
0
answers
215
views
positive and negative sample count for ConvNets [duplicate]
I have been trying to set up a ConvNet to classify some data. This data should be classified to either 1 (being what I need to get from the image) and 0 for everything that is irrelevant. I have ...
- 11
1
vote
0
answers
162
views
While dealing with imbalanced classes, to what extent can we upsample a minority class? [duplicate]
I have my training data with the following approximate distribution:
Negative events : 90,000
positive events : 5,000
Training a model would require to oversample the minority class (and might also ...
- 85
0
votes
0
answers
143
views
Handling imbalanced data for classification [duplicate]
What are the best ways to deal with imbalanced datasets for classifying whether or not individuals pay their tuition? The data is 75% positive class (paid) and 25% negative (unpaid). Some approaches I ...
- 1,030
3
votes
0
answers
130
views
Unbalanced groups and classification errors [duplicate]
I would like to adopt a general strategy for dealing with an very unbalanced dataset, where my "positive" group corresponds to 1/40 of all the observations. The reason why I ask it is because all the ...
- 561