Linked Questions

1
vote
1answer
166 views

How to deal with a highly unbalanced classification problem?

I currently have data where there are 5 labels, 1,2,3,4,5 for my $Y$ variable and a set of associated predictors $X$. The problem is, I have around $10000$ observations with label $1$ and around ...
0
votes
1answer
122 views

oversampling in nested cross validation

Introduction I have a small mixed dataset consisting of continuous and categorical independent variables with a dichotomous dependant variable. I'm running various algorithms (neural networks, random ...
0
votes
0answers
188 views

How to improve specificity with unbalanced data? (R caret package)

I am working on a classification problem where my outcome variable is either "Approved" or "Denied". The % of approvals in my dataset is roughly 60% and the denials make up roughly 30%. I have tried ...
0
votes
0answers
188 views

Balanced LogLoss with XGBoost

Following the discussion on here I started worrying less about class imbalance. However, I recently started building a predictor, using XGBoost, and I wanted to used LogLoss as my target metric. I ...
1
vote
0answers
64 views

How do you deal with imbalanced data when you're doing regression?

To describe my problem. I'm predicting the price of an item depending on some text and other features in an ad. The training data contains a bunch of cheap items, some medium price items and few ...
0
votes
2answers
49 views

Population / Sample question

Today we started arguing at work, and couldn't come to conclusion. Let's say that we have a population of 1000 observations about various people. 50 of these people went bankrupt (1 - bankrupt, 0 - ...
0
votes
1answer
34 views

Dealing with dataset imbalance: test if adjusting is necessary

I'm currently working on a project which uses a imbalanced dataset (two classes) for training, and I'm not sure if I should do a resampling procedure or not. Is there a way to actually test if it's ...
0
votes
0answers
38 views

Training set target categories' distribution [duplicate]

In a book I'm reading I've come across the following quote: Accuracy on the test set is a good performance measure only when there is a relatively uniform distribution of target categories in the ...
0
votes
1answer
26 views

How to research very unbalanced groups

I have a data set for a PhD paper in which I need to look for differences between 3 groups of observations. The issue is that ~95% are in one group and the rest are in the other two. The data table is ...
1
vote
0answers
28 views

Running SMOTE on very large class imbalanced datasets - batched or subsampled implementations

There is a theoretical and computational aspect to this question. I was trying to use SMOTE to reduce class imbalance in a rather large dataset--about 8 million rows. The data has a binary outcome ...
0
votes
1answer
24 views

Binary Classification problem for imbalanced dataset

I am new to machine learning and need help. I have a dataset with two classes(0,1) where is 0 is Profitable and 1 is Unprofitable. Ratio of 0:1 in train set is 150/52 Taking positive as "1"(...
0
votes
1answer
22 views

Should the validation set have the same ratio in the categories as the whole data?

I'm currently working on a classification problem. The variable Y in 70% of cases is 0 and in 30% of cases is 1. Does my validation set have to have this same proportion? I ask because after using ...
0
votes
0answers
22 views

Is matching necessary for prediction from logistic regression?

I have a data set with very imbalanced groups (about 3% have an effect, while 97% don't). I experimented with logistic regression and was convinced that I can get to a very good accuracy of prediction ...
1
vote
0answers
17 views

When is oversampling preferable to undersampling and vice versa?

When data is unbalanced, that is, when the distribution of classes being predicted is very uneven (e.g. 90%/10% for two classes or 10%/15%/75% for three classes), many machine learning models have ...
0
votes
0answers
12 views

Random sampling for both under- and over-representative class

I have an unbalanced dataset. Let's say I have 500 positives and 50,000 negatives. Can I deal with this by randomly choose 300 out of 500 positives and also 300 out of 50,000 negatives? Does this ...

15 30 50 per page