Data organized into discrete categories or *classes* may present problems for certain analyses if the number of observations ($n$) belonging to each class is not constant across classes. Classes with unequal $n$ are *unbalanced*.

learn more… | top users | synonyms

0
votes
1answer
8 views

Permutation on multiple Pairwise Comparison with least square means in R 3.1.2

I'm working on a study where I employed an analysis of covariance (Ancova) with unbalanced factors. I used permutation tests to obtain p-values for my estimates since my observation does not come from ...
0
votes
0answers
6 views

comparing probability of imbalance classes

I am trying to figure out what id the standard way to compare the probability of occurrence of two imbalances classes: let say there 1500 web pages with different languages edition (e.g. Wikipedia). ...
0
votes
0answers
20 views

cost matrix, unbalanced class, oversampling and threshold probability

Let's suppose I have a cost matrix with TP=+90 FP=-10 TN=0 and FN=-10, and that the class is unbalanced. I need to capture the costs in my decision. To do so, I always consider the probability ...
0
votes
1answer
16 views

Binary classification in imbalanced data

I have a binary classification problem where my data is highly imbalanced (80:20). Given this imbalance is present in both training and test set, does it make sense to apply specific strategies during ...
0
votes
0answers
26 views

SVM One-vs-One vs One-vs-ALL SVM

For an unbalanced dataset annotated by human annotators in which each item is assigned to different classes, what is the argument for and against using any of One-vs-One vs One-vs-ALL SVM ...
2
votes
2answers
62 views

Does Support Vector Machine handle imbalanced Dataset?

Does SVM handles imbalanced dataset? Is that any parameters (like C, or misclassification cost) handling the imbalanced dataset?
0
votes
1answer
54 views

Which post-hoc is more valid for multiple comparison of an unbalanced lmer-model: lsm or mcp?

After doing a model comparison with my mixed lmer model, I have a model with three main effects, no interaction, say ...
1
vote
1answer
37 views

LSmeans - Unbalanced data with interactions

I wish to analyze an unbalanced data set with 3 variables Tleaf, Tair, and orientation (factor with two levels). Considering the effect of the factor "orientation", I wish to determine if "Tair" has a ...
1
vote
0answers
82 views

Any advice on how to improve my accuracy rate in text classification?

I'm trying to do a text classification task. Here are some specs: Context file size = 1M+ documents already labeled Number of top-labels = 17 Number of sub-labels = around 130 Each document is ...
0
votes
0answers
22 views

dealing with imbalanced data set in multiclass text classification

I need to build a text classification model. I have a labeled training set and my goal is to classify the new unlabeled text . My training set is composed on 6 categories, that are imbalanced. The ...
0
votes
0answers
36 views

unary classification in PyBrain

I've just started using PyBrain for some data classification work, and I've gotten it working pretty well where I have data from all possible classes and I can train the network using all the classes. ...
0
votes
0answers
24 views

Unbalanced groups in a Hierarchical Linear Model

I would appreciate some practical and/or conceptual advice on sample sizes in a two level hierarchical linear model. A lot of the material on sample sizes and HLM is about the number of level 2 ...
1
vote
0answers
9 views

Multiclass target detection : N X (1 vs all) or 1 X (N vs all) ?

I am doing a multiclass classification using neural networks. say I have 10 target classes and one null (non-of-the-above-targets). is it better that I train a neural network separately for each ...
1
vote
2answers
48 views

If one group has 23 participants and the second group has 117 participants in it, can we rely on the result of t-test? [duplicate]

I was evaluating a research paper in which the authors have 23 male participants and 117 female participants. They have applied t-test to calculate Gender differences and have even concluded on ...
2
votes
1answer
70 views

Has anyone publicly shared an implementation of RUSBoost in R?

There's no package available on CRAN, so I was hoping someone in the community had written their own function/package. I see it's been done in MATLAB, so I may just have to start with that and write ...
2
votes
0answers
73 views

Average of predicted values with logistic regression

I have a large unbalanced dataset (the target has ~1500x more 0's than 1's) on which I train a logistic regression algorithm to predict the probability of success (Not a binary outcome but a real ...
2
votes
0answers
71 views

Unbalanced panel data: Fixed effects?

I have an unbalanced panel dataset with N=10 firms and T=61 days. Because one variable had values outside the theoretical range I had to constrain my dataset, which left me with only 239 observations. ...
0
votes
0answers
18 views

How to analyze data with unequal observations per cell?

I've got a very poorly designed study, and I need to find a way to analyze the data I've collected. Here's a description of the design: There are two within-subjects variables, one with four levels ...
0
votes
1answer
81 views

How to avoid random forest overfitting and improve prediction?

I have an input dataset x_train and an output dataset y_train ...
1
vote
3answers
75 views

Significant main effect in one-way, but not in two-way, MANOVA

I used a one-way MANOVA to study the effect of age groups on the average of height and weight and found that they were significant. Then I used a two-way MANOVA to study the effect of age groups and ...
2
votes
2answers
103 views

Which classifiers work well with unbalanced data?

I have a binary classification problem which is very unbalanced - it can have 98% of data from one class. Which classifiers work well with this sort of data? I have an unlimited supply of training ...
1
vote
0answers
20 views

Achieving high recall for smaller class in unbalanced linear svm

I have an svm-related question. I have an unbalanced dataset, meaning classA could be 1/10 to 1/35 of classB. Well I am interested in getting a linear svm which would separate the data and would ...
1
vote
0answers
36 views

How to do a power analysis for an unbalanced mixed effects ANOVA?

I need suggestions for how to calculate the $n$ required for 80% and 90% power for >30% change from baseline ($T_0$) at $T_1$ or $T_2$, drug vs. placebo, 2:1 ratio; 20% CV in test; factors: subject, ...
0
votes
2answers
67 views

How to deal with a skewed class in binary classification having many features?

I am doing data analysis in the mobile ad targeting domain. I have around 18 features and for a combination of these features, the result is either True or False (1/0) depending on whether the ...
0
votes
1answer
47 views

Unbalanced test data matter?

I have balanced training data: 300 positives and 300 negatives, but the test data is unbalanced: I have 15 positives and 60 negatives. Will the unbalanced test data impact classification accuracy? ...
0
votes
0answers
91 views

Training and testing on Unbalanced Data Set

I used SMOTE algorithm in R for class balancing. My data size has 13000 rows, I had 7% minority class in my sample now I used SMOTE( Synthetic Minority Oversampling Technique) for class balancing such ...
1
vote
1answer
93 views

Logistic sample and case numbers

I have some questions about binary logistic regression. For my research, I am planning to use 12 predictors, and my sample consists of 129 cases. However, I know of a 1 to 10 rule. Additionally, my ...
1
vote
2answers
164 views

How to deal with unbalanced data

I'm doing data analysis with a dataset of 11795 data points (with 88 features). 85% (9973 points) of these data points correspond to data points belonging to class 1, 5% (589 points) belong to class 2 ...
0
votes
0answers
17 views

Is it correct to compare two distributions of observed (that are means of observation) and expected values with chi square?

I have 2 classification methods that I want to compare. So I ran them on the same dataset and I obtained 2 different classifications. Then, I want to test if their classification is robust and how. ...
1
vote
0answers
55 views

Oversampling with categorical variables

I would like to perform a combination of oversampling and undersampling in order to balance my dataset with roughly 4000 customers divided into two groups, where one of the groups have a proportion of ...
0
votes
0answers
19 views

the effect of unbalanced classes

I have a data set with label "Y" and "N", which is splitted into training and testing sets. The ratio the labels in training and testing sets are 3:1 and 1:1, respectively. If I build model using the ...
0
votes
1answer
69 views

Improving SVM classification

I have a classification problem (bioinformatics domain) where I have around 333 features. Currently, I am first selecting features (using importance feature of random forest) and then pushing the same ...
0
votes
0answers
25 views

How to account for multiple & varying amounts of observations per factor level & still retain info? As a random effect in GLMM or take the mean?

I would appreciate any help! Specifically I would like to know which option is best. Question Does Var2 influence Var1 in relation to the factor(Ind), and does Var3 & Var4 also have some effect. ...
0
votes
0answers
44 views

How to handle imbalanced data using SMOTE-N (nominal data) to generate multiple synthetic data?

I have a classification problem with two classes working on nominal data. I want to apply SMOTE-N to deal with imbalanced data. However, it is not clear to me how to use SMOTE-N for generating N ...
0
votes
0answers
504 views

One-way ANOVA with unequal sample sizes

I have a function for performing one-way ANOVA in R: ...
1
vote
1answer
68 views

Is f-measure synonymous with accuracy?

I understand that f-measure (based on precision and recall) is an estimate of how accurate a classifier is. Also, f-measure is favored over accuracy when we have an unbalanced dataset. I have a simple ...
1
vote
0answers
62 views

Testing classifier with binomial test when group sizes are unequal

I have data from 50 human subjects, who are divided into groups A and B (30 participants are in group A and 20 participants in group B). I also have a range of measurements from each subject. I have ...
0
votes
0answers
22 views

Comparing networks of groups of unequal size

I want to compare some network properties such as entropy, connectivity, etc., between 2 groups. I have time series fMRI data on patients (n = 80) and controls (n = 30) for certain regions of the ...
2
votes
3answers
234 views

Suggestions for cost-sensitive learning in a highly imbalanced setting

I have a dataset with a few million rows and ~100 columns. I would like to detect about 1% of the examples in the dataset, which belong to a common class. I have a minimum precision constraint, but ...
0
votes
2answers
202 views

Handling unbalanced data using SMOTE - NO BIG DIFFERENCE?

I have a classification problem with 2 classes. I have nearly 5000 samples, each of which is represented as vector with 570 features. The positive class samples are nearly 600. Meaning, I have a 1:8 ...
0
votes
0answers
60 views

Repeated Measures ANOVA or mixed model? And with massive group differences?

I need some guidance on which statistical test to use: I want to know if the nature of frog calls changes depending on the time of day. I observed 4 species of frogs at the same 3 times for several ...
0
votes
2answers
81 views

Determine if difference in class distribution is statistically significant

I have a dataset of some observations with class attribute of values 0 and 1. The dataset is quite unbalanced (class 1 – 15%, class 0 – 85%). Further this dataset consists of 5 years, and the ...
2
votes
1answer
165 views

Which metric should I trust to evaluate my predictive model

I am working on predictive model and when I evaluate it, I find good accuracy_score, precision_score, ...
0
votes
0answers
35 views

Trial with two sites, two years - Different number blocks - Problem? (Q1)

I have got two questions on an agricultural field trial that was conducted at two sites in two conscutive years. Virtually everything was the same in all trials (crop variety, planting density...). ...
1
vote
3answers
252 views

SVM for unbalanced data

I want to attempt to use Support Vector Machines (SVMs) on my dataset. Before I attempt the problem though, I was warned that SVMs dont perform well on extremely unbalanced data. In my case, I can ...
2
votes
2answers
59 views

Collecting training data for document classification with unbalanced classes

I have a document classification problem in which the estimated class proportions in the population are severely unbalanced: the population is ~99% class 0 and ~1% class 1. I am using a logistic ...
0
votes
0answers
55 views

How do I partition variance among nested random effects for non-normal data and an unbalanced design?

I have a dataset of plant drought tolerance values (called TLP_DRY) that I would like to partition variance for among the nested levels Biome/Study site/Species to figure out whether most variation in ...
0
votes
1answer
169 views

Should I specify Prior or Cost matrix with Tree Bagger in Matlab

I'm trying to create Random Forests in Matlab and there are more observations in some classes than there are in others. Do I need to specify this as a cost matrix or as a prior probability or will ...
3
votes
1answer
161 views

Using Wilcoxon-Mann-Whitney test for comparing two population of different sizes

I am using Wilcoxon-Mann-Whitney test for comparing two populations. Unfortunately, sizes of my population are different; one has size 100000 and the other 6000. Can I use this test to compare these ...
2
votes
1answer
186 views

unbalanced samples random Forests

I am trying to predict species presence or absence using randomForest in R (classification). In fact, I am trying to do it for several species, in separate models. For a couple of the species, the ...