Questions tagged [stratification]

A sampling technique in which the population of interest is partitioned into subsets ("strata") based on characteristics known at all units before sampling.

169 questions
13k views

Benefits of stratified vs random sampling for generating training data in classification

I would like to know if there are advantages of using stratified sampling instead of random sampling, when splitting the original dataset into training and testing set for classification. Also, does ...
71 views

Stratification based on propensity score and a second variable

The basis of my question is whether it makes sense, given you have a set of propensity scores, to stratify the data based on the propensity scores and on a second variable which was included in the ...
29 views

What effect does the weight function have in the stratified population mean formula?

The stratified population mean formula is $\bar{X}=\sum_{h=1}^HW_h\bar{X}_h$, where $W_h=\frac{N_h}{N}$ is called a weight function. Can someone explain intuitively what affect this weight function ...
116 views

How calculate Gini coefficient (unbiased) and Lorenz Curve with stratified sample?

I'm working with a stratified sample by minimum variance method with 3 levels that becomes at 1176 stratum. The sample comes from finite population of income taxes data. I'm looking for how calculate ...
131 views

How can i Age adjust the proportions of my covariates in a multiple stratified data set?

I am doing a cross sectional analysis in a huge data set. The initial descriptive analysis is stratified by gender, 4 calendar periods and smoking status which gives me the proportions within the ...
7k views

How to stratify a dataset to keep groups of data together in Python?

I have a dataset I want to use for machine learning that looks something like this: ...
375 views

Horvitz-Thompson variance estimation when estimating across strata

I have a sample of Business units, which has been stratified according to two stratification variables (Revenue class and field of Business acitivity). Within the strata, Units were sampled according ...
99 views

Combined analysis?

We have performed analyses by dividing the participants into 6 groups. The attribution of the groups was based on two variables: the Body Mass Index (BMI) and Metabolic syndrome (MetS): if BMI<25 ...
431 views

Election fraud detection: the statistics of Quick Count

I’m reading the book Quick Count and Election Observation (chapter 5). I’m interested in understanding the statistics used in Quick Counts. Quick Counts is a methodology for verifying official ...
120 views

calculate properly statistical moments with stratified sample data

I have problems of how to calculate properly the third and fourth order moments: my data is a stratified sampling with three strata. The goal for me is to make a descriptive analysis: mean, variance, ...
36 views

Ideas for sampling design

I have a data base of schools per district and the number of teachers per school. I want to select a sample where between 1 an 10 schools are selected in each district. Once a school is selected, the ...
2k views

Using post-stratification weights in R survey package

I am analyzing a dataset that has a variable for post-stratification weights. As this is a complex survey, the plan is to use the R survey package. I have been ...
656 views

Post stratification weights and the survey package in R

I have survey data that needs to be weighted, and to help me with this task, I have access to the full joint distributions of the variables I want to use. As I understand it, I should use the ...
82 views

How to calculate a sample size in the case of a stratified 3 degrees sample design ?

I am trying to calculate the sample size (and the repartition between PSUs and SSUs) needed to obtain an estimation of a proportion (that I assume to de around 50%) with a 0.05 standard error in the ...
222 views

Confidence intervals on weighted proportions

I am conducting a survey that will identify and quantify the extent to which businesses in a certain industry reuse, recycle and dispose of their waste. I am trying to figure out how I would be able ...
367 views

Stratified sampling when creating a test/training set

I am creating a dataset that I am going to use as the training/test sets for a supervised classification problem. The problem is to identify quotations in a large corpus. I have randomly sampled the ...
42 views

Is there a method of stratum selection based on minimising variance within strata?

I have a population of about 7200 businesses from which to sample 2100 for a survey. The sample is to be stratified, but I have no information whatsoever on the usual way to stratify this population, ...
2k views

Neyman Allocation Standard Deviation for Proportions

I am currently working towards the Royal Statistical Society's Higher Diploma and have run across a strange result in one of their sample papers that I can't work out - wonder if anyone can help me. ...
201 views

What is the procedure to choose which strata to use if you have multiple strata available?

I have multiple characteristics of a population available (gender, age, place), and I have the answer for each individual for a particular yes-no question available done previously. For example, here ...
582 views

K-fold in grid-search for linear svm C parameter giving same value?

So every time I do a GridSearchCV with KFold, stratified or not, I get the same accuracy score and STDev for values of C=1,C=10, and C=100. I then did a special ...
137 views

Can interactions save power in a regression (vs. stratification)?

I have data with about 700 observations total. There are 9 subgroups in my sample with sizes ranging from 54-96. I want to use linear regression to explore how 8 independent variables might predict ...
118 views

Downsides of stratified randomization in experimental design

When sample sizes are too small to trust that the usual asymptotics will guarantee good balance across experimental groups on known confounders, a common approach is stratified or blocked ...
168 views

dataset shift and adjustment in glmnet

I have two sets of data A and F; In each datasets, I have two types of data point, say, 0 and 1 and the goal is to make a classifier. One strategy is training/fitting the model, using one dataset, ...
632 views

How to avoid overfitting in stratified k-fold cross validation methods for classification?

I'm using stratified 10-fold CV on my data set that is imbalanced. I've read in articles that this method is useful for such data set. But I'm not sure if I'm using this method as well. In each fold, ...
15k views

Why use stratified cross validation? Why does this not damage variance related benefit?

I've been told that is beneficial to use stratified cross validation especially when response classes are unbalanced. If one purpose of cross-validation is to help account for the randomness of our ...
87 views

Small Random Samples vs one Random Sample (Stratified)

Backdrop - I will be doing stratified random sampling from a data that is about 100 million events. Distribution of this original data is extreme long tail (1% of objects contribute to 95% of events). ...
780 views

Using strata instead of stratifying groups separately?

Can someone help me understand what strata(groups) does when it is used as a covariate? I am used to seeing stratified models where each level is run as a separate model, so that you can get ...
607 views

Propensity score stratification: standard errors and p-values

While there are many tutorials on how to perform propensity score stratification, I was unable to find any example that showed the calculation of standard errors and p-values for the final estimate. ...
77 views

What does it mean if the ROC scores are quite different when using the Stratified K fold with and without shuffling?

I'm currently building a random forest classification and trying to measure the model performance by the [mean ROC area]. With the same data set: When I use cross_validation.StratifiedKFold(y, ...
519 views

How to estimate survey data at cluster level in a stratified clustering survey design?

While estimating from the survey data involving stratification & clustering survey design and using survey package of r, is it possible to estimate at the cluster level? For eg; for following ...
6k views

Sampling with replacement in R randomForest

The randomForest implementation does not allow sampling beyond the number of observations, even when sampling with replacement. Why is this? Works fine: ...
384 views

How to partition a training-set when I have a big class imbalance?

In my actual data class A has 90%, class B has 9% and class C has 1% (numbers are made up for sake of simplicity). Now I want to prepare a training set for my classifier (I plan to use Vowpal Wabbit). ...
662 views

Different “end of study” times for different cohorts - Cox PH model in survival analysis

I have a dataset with 4 cohorts of about the same size (~700 people each). I'm trying to apply a Cox PH model using the time needed to pass a very difficult exam as my "time" variable. The cohorts ...
65 views

Stratum on subset of data: drop if too small?

We have genotype data for ~20000 samples with 4 principal components and 10 strata - study centres. Formula for an example analysis would be: ...
2k views

Calculating needed sample size for two stage stratified cluster sample

We want to carry out a survey to assess perceptions of quality and access to medical education of last year students. Based on our research, we have decided that a two stage stratified cluster sample ...
91 views

Data stratification for long-tailed distributions

I'm encountering a few problems successfully stratifying a dataset that is long-tailed to the right. In my research, I came across this excellent response: Sampling from a long tailed distribution ...
45 views

Can unintentional sample stratification be problematic?

An example of what I mean: I have a certain essay from all students at a university. I take a 1% random sample (not stratified) and run some time consuming computational analysis on each essay ...
3k views

How to conduct Cox PH stratisified by matched pair in R

I am wanting to calculate hazard ratio in a matched cohort design. For such I am using the coxph() function in R. But I have been recommended to stratify by matched ...
2k views

Confidence interval for a proportion estimated through stratified sampling

When estimating the confidence interval for a binomial proportion $p$, with $\hat{p}$ near 0 or 1, one has to use something other than the Wald interval to get a reasonable estimate (see for instance ...
1k views

Is stratified meta-analysis more or less objective than meta-regression?

Reviewer asked me why I use meta-regression as a way how to deal with heterogeneity among effect sizes instead of conducting stratified meta-analysis. I tried to google "stratified meta-analysis" and ...
506 views

Ignore strata in external validation of stratified Cox prop hazards model?

I've fit a stratified Cox proportional hazards model to some survival data, where I've stratified by a potential confounder which is the batch the data comes from (there are three batches). Now, I'd ...
514 views

Deriving the formula for the mean of a stratified sample

The formula for the mean of a stratified sample $\bar Y_s$ is: $$\bar Y_s = \frac 1 N \sum_i N_i \bar Y_i$$ where $N$ is the sample size for all strata, and $N_i$ and $Y_i$ are the sample size and ...
57 views

which statistical sampling technique to employ to have a representative sample

There is kwashiorkor outbreak in my community Abeng (a city in Africa). The community is in sections A to F and it heterogeneous in nature.
1k views

Stratified or multi-stage sampling?

I just started volunteering with a small organization that wants to estimate the average and total number environmental trouble spots in a large US city. They have a clear definition of an ...
95 views

I'm comparing various conditions versus some disease for a large stratified/clustered dataset which purports to account for the entire population after weighting -- hence the use of RR instead of OR. ...
2k views

Individual and overall RMSE for multivariate data

I have a dataset which contains missing values, and I'm using imputation packages (Rs mi and ...
361 views

Stratified random sampling and its distribution

I'm learning stratified random sampling and I'm having hard time to solve this problem. consider the following population of six units: ...
626 views

Stratified Cross-Validation with Collaborative Filtering

My dataset consists of binary preferences ($0$ or $1$) given by users on items like this: User-ID | Item-ID | Preference If a user has not given a preference to an item, then it is not in the ...