Linked Questions

1
vote
0answers
2k views

Repeating rare examples in unbalanced data classification [duplicate]

So I'm trying to train a neural network for a rare event detection. Based on that, I have like 1000 times more examples for non-target (everything else) examples that I have for target examples. So I ...
16
votes
0answers
337 views

Are unbalanced datasets problematic, and (how) does oversampling (purport to) help? [duplicate]

TL;DR See title. Motivation I am hoping for a canonical answer along the lines of "(1) No, (2) Not applicable, because (1)", which we can use to close many wrong questions about unbalanced datasets ...
0
votes
1answer
479 views

Does it make sense to up-sample imbalanced data if I care about predicting the right probabilities? [duplicate]

I'm developing a predictive model on a dataset of about 25K observations with the response variable having ~60 classes. There are ~130 predictor variables, all of them binary. In this problem I care ...
2
votes
1answer
187 views

When do we speak of imbalanced data? [duplicate]

I was wondering, what the threshold is for a dataset to be called 'imbalanced'? Technically every dataset where the target-classes aren't evenly distributed is not balanced. Yet small imbalance ...
2
votes
0answers
232 views

Why imbalance in the data set is an issue in data mining and machine learning? [duplicate]

I've analyzed a data set from a credit card company before and it has the famous "unbalanced classes" problem like all the other credit card companies, i.e., in the data set (the information of users ...
2
votes
2answers
79 views

Does the ratio of training data (significantly) affects the result? [duplicate]

Say I have two training data of email titles with 10000 entries each. One of them have 2000 ham and 8000 spam, while the other have 5000 ham and 5000 spam. Will predicting any random email with the 2:...
1
vote
0answers
182 views

Classification model on a highly unbalanced dataset [duplicate]

I’m dealing with a highly unbalanced dataset where 20% of data belongs to class A and 80% belongs to class B. It’s very hard for us to produce synthetic class A data. Just wondering if the below ...
1
vote
0answers
154 views

positive and negative sample count for ConvNets [duplicate]

I have been trying to set up a ConvNet to classify some data. This data should be classified to either 1 (being what I need to get from the image) and 0 for everything that is irrelevant. I have ...
0
votes
0answers
48 views

Handling imbalanced data for classification [duplicate]

What are the best ways to deal with imbalanced datasets for classifying whether or not individuals pay their tuition? The data is 75% positive class (paid) and 25% negative (unpaid). Some approaches I ...
0
votes
0answers
45 views

Choosing better samples for downsampling [duplicate]

I'm training a classification model for highly unbalanced data where hits are pairs that have similarity of almost 1 on independent metric. Everything else are non hits. Does it make sense to pick ...
1
vote
0answers
39 views

While dealing with imbalanced classes, to what extent can we upsample a minority class? [duplicate]

I have my training data with the following approximate distribution: Negative events : 90,000 positive events : 5,000 Training a model would require to oversample the minority class (and might also ...
0
votes
0answers
34 views

unbalaced data set in classification tasks [duplicate]

How do I judge whether the dataset is unbalanced (is it when the minority class inferior of 15%) could us use the test CR after balancing the data?
1
vote
0answers
23 views

Risks of disproportionate numbers of training examples for different classes? [duplicate]

fairly new to machine learning so please be gentle! I was wondering what the risks might be if I have significantly more examples for one class than the other when training a 2-D perceptron; for ...
0
votes
0answers
18 views

At which threshold is unbalanced data a problem for a binary classification tree? [duplicate]

I want to build a binary classification tree to clasfiy wether a person is working or not and use the model for prediction. I read that unbalanced data could be a problem. Now i ask myself at which ...
0
votes
0answers
16 views

Will more training data of one class destroy the good model? [duplicate]

I am facing a binary classification problem that I don't know I should use more data or not. I have one label 'A' with 10 training examples. And another label 'B' with also 10 training examples. By ...
1
vote
0answers
12 views

Multiple class prediction with unbalanced data [duplicate]

I have 4 different web sites (A/B/C/D) and need to predict whether a user will rather convert on A/B/C or D or not convert at all. My data set has 100000 rows and only 100 of those are converted user (...
0
votes
0answers
10 views

Do I need to treat class imbalance before preforming RFE (Recursive Feature Elimination)? [duplicate]

I am trying to perform RFE on an imbalanced data set that will later be used for binary classification. I have chosen to use the caret::rfe package for feature selection. I am using random forests ...
0
votes
0answers
9 views

Unbalanced classification problem [duplicate]

I am trying to build models for the KDD cup 2004 challenge. The protein homology problem data is divided into several blocks with roughly 1000 samples in it. Each block has an unbalanced class problem....
0
votes
0answers
9 views

I am building model for classification and in practice my data could be imbalance to any extend. How could I build single ml model to perform task? [duplicate]

for example: Case 1: Class A:10 Observations Class B: 100 Observation Case2: Class A: 100 Observations Class B: 10 Observations How could I build a single model (Random Forest) to perform this task?
100
votes
7answers
33k views

Why is accuracy not the best measure for assessing classification models?

This is a general question that was asked indirectly multiple times in here, but it lacks a single authoritative answer. It would be great to have a detailed answer to this for the reference. ...
80
votes
3answers
69k views

Does an unbalanced sample matter when doing logistic regression?

Okay, so I think I have a decent enough sample, taking into account the 20:1 rule of thumb: a fairly large sample (N=374) for a total of 7 candidate predictor variables. My problem is the following: ...
48
votes
4answers
22k views

Class imbalance in Supervised Machine Learning

This is a question in general, not specific to any method or data set. How do we deal with a class imbalance problem in Supervised Machine learning where the number of 0 is around 90% and number of 1 ...
40
votes
4answers
34k views

Training a decision tree against unbalanced data

I'm new to data mining and I'm trying to train a decision tree against a data set which is highly unbalanced. However, I'm having problems with poor predictive accuracy. The data consists of students ...
14
votes
3answers
17k views

SVM for unbalanced data

I want to attempt to use Support Vector Machines (SVMs) on my dataset. Before I attempt the problem though, I was warned that SVMs dont perform well on extremely unbalanced data. In my case, I can ...
29
votes
2answers
1k views

What is the root cause of the class imbalance problem?

I've been thinking a lot about the "class imbalance problem" in machine/statistical learning lately, and am drawing ever deeper into a feeling that I just don't understand what is going on. First let ...
13
votes
2answers
6k views

Bagging with oversampling for rare event predictive models

Does anyone know if the following has been described and (either way) if it sounds like a plausible method for learning a predictive model with a very unbalanced target variable? Often in CRM ...
16
votes
1answer
2k views

What problem does oversampling, undersampling, and SMOTE solve?

In a recent, well recieved, question, Tim asks when is unbalanced data really a problem in Machine Learning? The premise of the question is that there is a lot of machine learning literature ...
2
votes
2answers
3k views

Classification algorithms for handling Imbalanced data sets

I’m working on a classification problem where dataset is extremely imbalanced ( roughly 13000 "zero" and 100 "one" responses). As the first step, I trained a Logistic Regression and changing the ...
4
votes
1answer
674 views

Does class balancing introduce bias?

I have a data set that is imbalanced, the prediction rate is not much better than the base line without doing any class balance. I have two classes and I can't collect more data. What I have done: ...
1
vote
1answer
2k views

Regression on imbalanced data

I want to predict a continuous value between 0 and 1 and the true labels are 99% (out of 100000 samples) zero and rest of them are between 0 and 1. What are the approaches that I can take so that I ...
1
vote
0answers
913 views

Why would random forest perform bad on unbalanced class

There is a huge number of posts saying that an imbalanced classes are bad. And only half explains it in terms of recall-presicion scores, meaning that accuracy can be high but F1 score low. What I ...
1
vote
1answer
488 views

Calibration after up and downsampling

I am experimenting with different techniques to deal with imbalanced classes in a classification problem. I am comparing upsampling the minority class with downsampling the majority class. Furthermore ...
1
vote
0answers
333 views

Effects of class imbalance on nn batch training

Say I have a binary classification task, where the positive class (1) is only 1% of the whole data set. Intuitively I can understand why this could be bad for the classifier as the model may learn ...
1
vote
1answer
220 views

Imbalanced Test Data

I have an imbalanced (1:5) training and test set with only two classes and have oversampled the training set with SMOTE so that the class ratio is 1:1. The ML model gives values over 0.7 for accuracy, ...
1
vote
1answer
154 views

How to deal with a highly unbalanced classification problem?

I currently have data where there are 5 labels, 1,2,3,4,5 for my $Y$ variable and a set of associated predictors $X$. The problem is, I have around $10000$ observations with label $1$ and around ...
2
votes
0answers
220 views

How to work with unbalanced datasets in generalized Boosted Regression Models using gbm in r?

I'm using generalized boosted regression models to explore what is the contribution of 20 independent environmental variables (x1, x2, ...., x20) to the explanation of the variability of the dependent ...
1
vote
1answer
188 views

When to use stratified k-fold

According to a post on Analytics Vidhya: Having said that, if the train set does not adequately represent the entire population, then using a stratified k-fold might not be the best idea. In such ...
0
votes
1answer
104 views

oversampling in nested cross validation

Introduction I have a small mixed dataset consisting of continuous and categorical independent variables with a dichotomous dependant variable. I'm running various algorithms (neural networks, random ...
1
vote
0answers
163 views

balance_classes in H2O, but for regression? [closed]

I am training with deep learning for regression in H2O for R. My dataset is unbalanced (ie. not evenly distributed). There has been discussion on whether unbalanced datasets are an issue or not, with ...
1
vote
3answers
51 views

Machine Learning - How to Sample Test and Training Data for Rare Events

Suppose I have a data set with 1000 observations. I want to train and test a Classification Model to predict a target variable as true or false. However, in my observation set, true occurs only say 10%...
0
votes
0answers
135 views

Balanced LogLoss with XGBoost

Following the discussion on here I started worrying less about class imbalance. However, I recently started building a predictor, using XGBoost, and I wanted to used LogLoss as my target metric. I ...
1
vote
0answers
56 views

How do you deal with imbalanced data when you're doing regression?

To describe my problem. I'm predicting the price of an item depending on some text and other features in an ad. The training data contains a bunch of cheap items, some medium price items and few ...
0
votes
2answers
47 views

Population / Sample question

Today we started arguing at work, and couldn't come to conclusion. Let's say that we have a population of 1000 observations about various people. 50 of these people went bankrupt (1 - bankrupt, 0 - ...
0
votes
1answer
32 views

Dealing with dataset imbalance: test if adjusting is necessary

I'm currently working on a project which uses a imbalanced dataset (two classes) for training, and I'm not sure if I should do a resampling procedure or not. Is there a way to actually test if it's ...
0
votes
0answers
37 views

Training set target categories' distribution [duplicate]

In a book I'm reading I've come across the following quote: Accuracy on the test set is a good performance measure only when there is a relatively uniform distribution of target categories in the ...
0
votes
1answer
25 views

How to research very unbalanced groups

I have a data set for a PhD paper in which I need to look for differences between 3 groups of observations. The issue is that ~95% are in one group and the rest are in the other two. The data table is ...
0
votes
0answers
25 views

How to improve specificity with unbalanced data? (R caret package)

I am working on a classification problem where my outcome variable is either "Approved" or "Denied". The % of approvals in my dataset is roughly 60% and the denials make up roughly 30%. I have tried ...
0
votes
1answer
22 views

Should the validation set have the same ratio in the categories as the whole data?

I'm currently working on a classification problem. The variable Y in 70% of cases is 0 and in 30% of cases is 1. Does my validation set have to have this same proportion? I ask because after using ...
0
votes
0answers
22 views

Is matching necessary for prediction from logistic regression?

I have a data set with very imbalanced groups (about 3% have an effect, while 97% don't). I experimented with logistic regression and was convinced that I can get to a very good accuracy of prediction ...