Data organized into discrete categories or *classes* may present problems for certain analyses if the number of observations ($n$) belonging to each class is not constant across classes. Classes with unequal $n$ are *unbalanced*.

learn more… | top users | synonyms

10
votes
2answers
3k views

Training a decision tree against unbalanced data

I'm new to data mining and I'm trying to train a decision tree against a data set which is highly unbalanced. However, I'm having problems with poor predictive accuracy. The data consists of students ...
10
votes
1answer
1k views

Optimising for Precision-Recall curves under class imbalance

I have a classification task where I have a number of predictors (one of which is the most informative), and I am using the MARS model to construct my classifier (I am interested in any simple model, ...
7
votes
2answers
789 views

Does a big difference in sample sizes matter for an independent t-test?

There is a very confusing question in my mind. I have data, and would like to compare numeric scores between men and women. There is a big difference in those two groups: the number of men is 34, ...
3
votes
2answers
286 views

Named entity recognition and class imbalance

I have implemented Maximum-entropy Markov model (MEMM) for the Named entity recognition (NER) problem. I have four classes: geographical, people, material (book titles etc) and other. Class ...
1
vote
1answer
93 views

Logistic sample and case numbers

I have some questions about binary logistic regression. For my research, I am planning to use 12 predictors, and my sample consists of 129 cases. However, I know of a 1 to 10 rule. Additionally, my ...
5
votes
2answers
386 views

Classification problem using imbalanced dataset

I am working on a pattern identification/classification problem on an imbalanced dataset, with target to non target proportion in population approx as 1%:99%. There are around 0.5 million records in ...
5
votes
1answer
3k views

Best way to handle unbalanced multiclass dataset with SVM

I'm trying to build a prediction model with SVMs on fairly unbalanced data. My labels/output have three classes, positive, neutral and negative. I would say the positive example makes about 10 - 20% ...
2
votes
3answers
239 views

Suggestions for cost-sensitive learning in a highly imbalanced setting

I have a dataset with a few million rows and ~100 columns. I would like to detect about 1% of the examples in the dataset, which belong to a common class. I have a minimum precision constraint, but ...
3
votes
2answers
3k views

LibSVM cost weights for unbalanced data doesn't work

I have a dataset where the number of negative labeled values is 163 times the number of positive labeled values. That is, I have an unbalanced data set. I tried: ...
0
votes
2answers
71 views

How to deal with a skewed class in binary classification having many features?

I am doing data analysis in the mobile ad targeting domain. I have around 18 features and for a combination of these features, the result is either True or False (1/0) depending on whether the ...
2
votes
1answer
222 views

Proper order of variables in unbalanced ANOVA

If you have unequal sample sizes in cells, then the order in which you enter model terms changes your results for sequential or Type I SS. The first variable to enter the model is allocated its ...
2
votes
1answer
477 views

Are there problems with inference using linear regression on observational data with highly skewed distributions of predictor values?

I am using a linear regression model to perform inference on some observational data. The samples are from an observational study and highly skewed along some of the dummy variables in the regression. ...
1
vote
2answers
711 views

By using SMOTE the classification of the validation set is bad

I want to do classification with 2 classes. When I classify without smote I get: ...
0
votes
1answer
55 views

Which post-hoc is more valid for multiple comparison of an unbalanced lmer-model: lsm or mcp?

After doing a model comparison with my mixed lmer model, I have a model with three main effects, no interaction, say ...
0
votes
0answers
35 views

Trial with two sites, two years - Different number blocks - Problem? (Q1)

I have got two questions on an agricultural field trial that was conducted at two sites in two conscutive years. Virtually everything was the same in all trials (crop variety, planting density...). ...