Data organized into discrete categories or *classes* may present problems for certain analyses if the number of observations ($n$) belonging to each class is not constant across classes. Classes with unequal $n$ are *unbalanced*.

learn more… | top users | synonyms

0
votes
2answers
82 views

How to deal with a skewed class in binary classification having many features?

I am doing data analysis in the mobile ad targeting domain. I have around 18 features and for a combination of these features, the result is either True or False (1/0) depending on whether the ...
1
vote
1answer
87 views

Finding the best dataset for classification

I have 100 datasets. All of them have varying number of features. There are around 20,000 samples in each of them. Every $i$-th sample in the 100 datasets has the same label ($0/1$). The data is ...
0
votes
1answer
91 views

How to avoid random forest overfitting and improve prediction?

I have an input dataset x_train and an output dataset y_train ...
4
votes
0answers
271 views

Mixed-effects model for a strongly unbalanced design

I am somehow unsure on the best option to analyze these data. Here is my study case: The response variable is a morphometric measure, one for each individual. During 10 years, say 2000-2009, people ...
4
votes
0answers
251 views

Softmax regression bias and prior probabilities for unequal classes

I'm using Softmax regression for a multi-class classification problem. I don't have equal prior probabilities for each of the classes. I know from Logistic Regression (softmax regression with 2 ...
3
votes
0answers
186 views

Mixed ANOVA: small and unbalanced samples

I have to analyze two samples ($n_1=10, n_2=18$) in a design in which there is a between-subject factor (Groups: 2 levels) and a within-subjects factor (...
3
votes
0answers
240 views

Problem with classifier after using SMOTE to balance the data

We've ran into a problem while training a classifier on an unbalanced data set. The response is binary with 0 indicating 'non defaulter' and 1 indicating 'defaulter' (it's a credit scoring task). ...
2
votes
0answers
80 views

Average of predicted values with logistic regression

I have a large unbalanced dataset (the target has ~1500x more 0's than 1's) on which I train a logistic regression algorithm to predict the probability of success (Not a binary outcome but a real ...
2
votes
0answers
88 views

Unbalanced panel data: Fixed effects?

I have an unbalanced panel dataset with N=10 firms and T=61 days. Because one variable had values outside the theoretical range I had to constrain my dataset, which left me with only 239 observations. ...
2
votes
0answers
253 views

Repeated measures ANOVA for an experiment with missing values

I have an experiment where several subjects (subjects $= S_1,S_2,...,S_m $) were asked to perform a set of tasks (tasks $= T_1, T_2, T_3,...,T_n$) using both their left ($L$) and right ($R$) arms. ...
2
votes
0answers
209 views

How to use k nearest neighbours for binary classification with unbalanced classes?

I have relatively large (100k items) dataset which I need to split in two groups. So far I've tried knn and the results are not good mainly because I have disproportion in my training data: 90% of ...
2
votes
0answers
544 views

How to analyze an unbalanced within-subject ANOVA design?

I have data from an experiment involving 4 groups of subjects, 2 possible interventions first and 3 possible intervention at a 2nd point, repeated data measurements from each subjects multiple time ...
1
vote
0answers
99 views

Any advice on how to improve my accuracy rate in text classification?

I'm trying to do a text classification task. Here are some specs: Context file size = 1M+ documents already labeled Number of top-labels = 17 Number of sub-labels = around 130 Each document is ...
1
vote
0answers
9 views

Multiclass target detection : N X (1 vs all) or 1 X (N vs all) ?

I am doing a multiclass classification using neural networks. say I have 10 target classes and one null (non-of-the-above-targets). is it better that I train a neural network separately for each ...
1
vote
0answers
21 views

Achieving high recall for smaller class in unbalanced linear svm

I have an svm-related question. I have an unbalanced dataset, meaning classA could be 1/10 to 1/35 of classB. Well I am interested in getting a linear svm which would separate the data and would ...
1
vote
0answers
43 views

How to do a power analysis for an unbalanced mixed effects ANOVA?

I need suggestions for how to calculate the $n$ required for 80% and 90% power for >30% change from baseline ($T_0$) at $T_1$ or $T_2$, drug vs. placebo, 2:1 ratio; 20% CV in test; factors: subject, ...
1
vote
0answers
60 views

Oversampling with categorical variables

I would like to perform a combination of oversampling and undersampling in order to balance my dataset with roughly 4000 customers divided into two groups, where one of the groups have a proportion of ...
1
vote
0answers
65 views

Testing classifier with binomial test when group sizes are unequal

I have data from 50 human subjects, who are divided into groups A and B (30 participants are in group A and 20 participants in group B). I also have a range of measurements from each subject. I have ...
1
vote
0answers
121 views

Questionnaire analysis and grouping responses

I want to know if the creative process engagement (CPE) is related to creativity. I don't know how to analyze the questionnaire. I will be collecting data for CPE with Zhang and Bartal's (2010) ...
1
vote
0answers
79 views

Type I ANOVA tests not depending on the order of the factors

I have a dataset with two factors A and B and the following design (contigency table showing the number of individuals for each ...
1
vote
0answers
103 views

Different Mean Square partitions in an unbalanced bifactorial ANOVA (with random factor) between R and Statistica

I am trying to extract variance components for selection and chance in a bifactorial design with Generation as a fixed factor and Replicate as a random term, for early fecundity. Since I am using ...
1
vote
0answers
194 views

How to compare multiple groups of unblanced repeated measures non normal data?

I'm trying to compare three groups data. But the data set is about a new drug trial. The data set has these characteristics: Follow-up set. That is, after administration of the drug, a series of ...
0
votes
0answers
9 views

positive and negative sample count for ConvNets

I have been trying to set up a ConvNet to classify some data. This data should be classified to either 1 (being what I need to get from the image) and 0 for everything that is irrelevant. I have ...
0
votes
0answers
7 views

comparing probability of imbalance classes

I am trying to figure out what id the standard way to compare the probability of occurrence of two imbalances classes: let say there 1500 web pages with different languages edition (e.g. Wikipedia). ...
0
votes
0answers
28 views

cost matrix, unbalanced class, oversampling and threshold probability

Let's suppose I have a cost matrix with TP=+90 FP=-10 TN=0 and FN=-10, and that the class is unbalanced. I need to capture the costs in my decision. To do so, I always consider the probability ...
0
votes
0answers
30 views

SVM One-vs-One vs One-vs-ALL SVM

For an unbalanced dataset annotated by human annotators in which each item is assigned to different classes, what is the argument for and against using any of One-vs-One vs One-vs-ALL SVM ...
0
votes
0answers
29 views

dealing with imbalanced data set in multiclass text classification

I need to build a text classification model. I have a labeled training set and my goal is to classify the new unlabeled text . My training set is composed on 6 categories, that are imbalanced. The ...
0
votes
0answers
50 views

unary classification in PyBrain

I've just started using PyBrain for some data classification work, and I've gotten it working pretty well where I have data from all possible classes and I can train the network using all the classes. ...
0
votes
0answers
28 views

Unbalanced groups in a Hierarchical Linear Model

I would appreciate some practical and/or conceptual advice on sample sizes in a two level hierarchical linear model. A lot of the material on sample sizes and HLM is about the number of level 2 ...
0
votes
0answers
18 views

How to analyze data with unequal observations per cell?

I've got a very poorly designed study, and I need to find a way to analyze the data I've collected. Here's a description of the design: There are two within-subjects variables, one with four levels ...
0
votes
0answers
102 views

Training and testing on Unbalanced Data Set

I used SMOTE algorithm in R for class balancing. My data size has 13000 rows, I had 7% minority class in my sample now I used SMOTE( Synthetic Minority Oversampling Technique) for class balancing such ...
0
votes
0answers
21 views

Is it correct to compare two distributions of observed (that are means of observation) and expected values with chi square?

I have 2 classification methods that I want to compare. So I ran them on the same dataset and I obtained 2 different classifications. Then, I want to test if their classification is robust and how. ...
0
votes
0answers
19 views

the effect of unbalanced classes

I have a data set with label "Y" and "N", which is splitted into training and testing sets. The ratio the labels in training and testing sets are 3:1 and 1:1, respectively. If I build model using the ...
0
votes
0answers
25 views

How to account for multiple & varying amounts of observations per factor level & still retain info? As a random effect in GLMM or take the mean?

I would appreciate any help! Specifically I would like to know which option is best. Question Does Var2 influence Var1 in relation to the factor(Ind), and does Var3 & Var4 also have some effect. ...
0
votes
0answers
45 views

How to handle imbalanced data using SMOTE-N (nominal data) to generate multiple synthetic data?

I have a classification problem with two classes working on nominal data. I want to apply SMOTE-N to deal with imbalanced data. However, it is not clear to me how to use SMOTE-N for generating N ...
0
votes
0answers
562 views

One-way ANOVA with unequal sample sizes

I have a function for performing one-way ANOVA in R: ...
0
votes
0answers
22 views

Comparing networks of groups of unequal size

I want to compare some network properties such as entropy, connectivity, etc., between 2 groups. I have time series fMRI data on patients (n = 80) and controls (n = 30) for certain regions of the ...
0
votes
0answers
63 views

Repeated Measures ANOVA or mixed model? And with massive group differences?

I need some guidance on which statistical test to use: I want to know if the nature of frog calls changes depending on the time of day. I observed 4 species of frogs at the same 3 times for several ...
0
votes
0answers
36 views

Trial with two sites, two years - Different number blocks - Problem? (Q1)

I have got two questions on an agricultural field trial that was conducted at two sites in two conscutive years. Virtually everything was the same in all trials (crop variety, planting density...). ...
0
votes
0answers
58 views

How do I partition variance among nested random effects for non-normal data and an unbalanced design?

I have a dataset of plant drought tolerance values (called TLP_DRY) that I would like to partition variance for among the nested levels Biome/Study site/Species to figure out whether most variation in ...
0
votes
0answers
182 views

How to set parameters for SMOTE in WEKA?

How I can choose the most optimal parameters for SMOTE in WEKA. More specifically, how should I pick the nearest-neighbours and the SMOTE to apply? I have been using the amount of SMOTE that gives the ...
0
votes
0answers
75 views

How do you analyse an unbalanced dataset using dummy variables?

I have a returns (finance) dataset with 3 factors, each with 3 levels: industry (banking, retail, others), year (2010, 2011, 2012), and size (small, medium, large). Each factor has been coded as a ...
0
votes
0answers
128 views

Interpreting coefficients for random effects models with extremely unbalanced data

I'm currently working with a data set that has numerous samples collected over time at different sites in a study area, and I'm interested in detecting a trend over time for that area. I know that in ...
0
votes
0answers
28 views

Impact of biased sampling on classifier training ?

Let's imagine I have a very unbalanced dataset with 99.99% of 0 and 0.01% of 1 on the target variable. What I want to do is make a classifier for this target. Now imagine that this dataset is very ...
0
votes
0answers
65 views

Sample size for binary text classification

In my application as a Binary text classification, one class has around 36,000 sample and another one has around 300 samples. I under-sample the first class. So, each class will have ~ 300 samples. ...
0
votes
0answers
117 views

How to create the class range given 1 as the middle class, 3 as the highest value and 0.65 as the lowest value?

Good day. I know getting the class interval given 3 as the highest value and 0.65 as the lowest value is easy. Here's the catch, the distribution of the interval starts at 1 which is considered as the ...