Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

Questions tagged [stratification]

A sampling technique in which the population of interest is partitioned into subsets ("strata") based on characteristics known at all units before sampling.

Filter by
Sorted by
Tagged with
1
vote
0answers
86 views

How to calculate a sample size in the case of a stratified 3 degrees sample design ?

I am trying to calculate the sample size (and the repartition between PSUs and SSUs) needed to obtain an estimation of a proportion (that I assume to de around 50%) with a 0.05 standard error in the ...
2
votes
0answers
232 views

Confidence intervals on weighted proportions

I am conducting a survey that will identify and quantify the extent to which businesses in a certain industry reuse, recycle and dispose of their waste. I am trying to figure out how I would be able ...
1
vote
0answers
388 views

Stratified sampling when creating a test/training set

I am creating a dataset that I am going to use as the training/test sets for a supervised classification problem. The problem is to identify quotations in a large corpus. I have randomly sampled the ...
0
votes
0answers
49 views

Is there a method of stratum selection based on minimising variance within strata?

I have a population of about 7200 businesses from which to sample 2100 for a survey. The sample is to be stratified, but I have no information whatsoever on the usual way to stratify this population, ...
1
vote
0answers
210 views

What is the procedure to choose which strata to use if you have multiple strata available?

I have multiple characteristics of a population available (gender, age, place), and I have the answer for each individual for a particular yes-no question available done previously. For example, here ...
0
votes
1answer
2k views

Neyman Allocation Standard Deviation for Proportions

I am currently working towards the Royal Statistical Society's Higher Diploma and have run across a strange result in one of their sample papers that I can't work out - wonder if anyone can help me. ...
2
votes
1answer
1k views

Splitting Longitudinal Data into Training & Test Sets [closed]

I'm trying to find a simple way to split some longitudinal data into a training and test set. I'm familiar with using the Caret package in R to make stratified splits, but only with wide-form data. It ...
0
votes
1answer
919 views

Estimation of sample size for stratified randomized clinical trial with R

We want to conduct a multi-center randomized clinical trial with two arms (2 treatments : a superiority trial) stratified on three centers (three sites). The primary outcome variable is a continuous ...
0
votes
1answer
602 views

K-fold in grid-search for linear svm C parameter giving same value?

So every time I do a GridSearchCV with KFold, stratified or not, I get the same accuracy score and STDev for values of C=1,C=10, and C=100. I then did a special ...
1
vote
1answer
154 views

Can interactions save power in a regression (vs. stratification)?

I have data with about 700 observations total. There are 9 subgroups in my sample with sizes ranging from 54-96. I want to use linear regression to explore how 8 independent variables might predict ...
4
votes
1answer
128 views

Downsides of stratified randomization in experimental design

When sample sizes are too small to trust that the usual asymptotics will guarantee good balance across experimental groups on known confounders, a common approach is stratified or blocked ...
0
votes
1answer
683 views

How to avoid overfitting in stratified k-fold cross validation methods for classification?

I'm using stratified 10-fold CV on my data set that is imbalanced. I've read in articles that this method is useful for such data set. But I'm not sure if I'm using this method as well. In each fold, ...
1
vote
0answers
173 views

dataset shift and adjustment in glmnet

I have two sets of data A and F; In each datasets, I have two types of data point, say, 0 and 1 and the goal is to make a classifier. One strategy is training/fitting the model, using one dataset, ...
2
votes
1answer
93 views

Small Random Samples vs one Random Sample (Stratified)

Backdrop - I will be doing stratified random sampling from a data that is about 100 million events. Distribution of this original data is extreme long tail (1% of objects contribute to 95% of events). ...
2
votes
1answer
2k views

How to weight data according to multiple variables

I have two datasets. Dataset A is considered the population. It has 7 variables with different number of classes / breaks for each. Dataset B is the dataset I want to make look like Dataset A in ...
3
votes
1answer
739 views

Reweighting sample to reflect census?

I'm working on a revision of a manuscript. I have a longitudinal cohort of adolescents that were followed throughout junior high school. One of the reviewers considers the attrition to be considerable,...
3
votes
1answer
925 views

Using strata instead of stratifying groups separately?

Can someone help me understand what strata(groups) does when it is used as a covariate? I am used to seeing stratified models where each level is run as a separate model, so that you can get ...
2
votes
2answers
239 views

Why class stratified sampling is not compatible for naïve bayesian modeling, if sampling is used?

I read a book recently and it mentioned related to prior probability of naïve bayesian: "Since the probability of an outcome is calculated from the data set, it is important that the data set used ...
2
votes
1answer
3k views

Standard Error for Sum

I am designing an algorithm for a stratified sampling on a population and then I want to find out what is the error bound for 95% confidence interval, for different sample statistic such as sum of the ...
1
vote
1answer
643 views

Propensity score stratification: standard errors and p-values

While there are many tutorials on how to perform propensity score stratification, I was unable to find any example that showed the calculation of standard errors and p-values for the final estimate. ...
1
vote
0answers
77 views

What does it mean if the ROC scores are quite different when using the Stratified K fold with and without shuffling?

I'm currently building a random forest classification and trying to measure the model performance by the [mean ROC area]. With the same data set: When I use cross_validation.StratifiedKFold(y, ...
3
votes
1answer
790 views

Two sample tests for stratified randomization

I have to test the difference between two samples which were selected via stratified randomization. The problem is that the samples are not independent due to randomization was stratified. How should ...
2
votes
0answers
257 views

Correlation with stratified data structure

I have a dataset with two stratas, the first level is called block, which represents the census block. the second level is two predetermined subgroups in each blocks, each subgroup contains several ...
0
votes
0answers
538 views

How to estimate survey data at cluster level in a stratified clustering survey design?

While estimating from the survey data involving stratification & clustering survey design and using survey package of r, is it possible to estimate at the cluster level? For eg; for following ...
13
votes
1answer
1k views

Does fitting Cox-model with strata and strata-covariate interaction differ from fitting two Cox models?

In Regression Modeling Strategies by Harrell (second edition) there is a section (S. 20.1.7) discussing Cox models including an interaction between a covariate whose main effect on survival we want to ...
2
votes
2answers
726 views

Different “end of study” times for different cohorts - Cox PH model in survival analysis

I have a dataset with 4 cohorts of about the same size (~700 people each). I'm trying to apply a Cox PH model using the time needed to pass a very difficult exam as my "time" variable. The cohorts ...
3
votes
0answers
65 views

Stratum on subset of data: drop if too small?

We have genotype data for ~20000 samples with 4 principal components and 10 strata - study centres. Formula for an example analysis would be: ...
2
votes
0answers
2k views

Calculating needed sample size for two stage stratified cluster sample

We want to carry out a survey to assess perceptions of quality and access to medical education of last year students. Based on our research, we have decided that a two stage stratified cluster sample ...
5
votes
2answers
3k views

Using post-stratification weights in R survey package

I am analyzing a dataset that has a variable for post-stratification weights. As this is a complex survey, the plan is to use the R survey package. I have been ...
0
votes
1answer
30 views

What effect does the weight function have in the stratified population mean formula?

The stratified population mean formula is $\bar{X}=\sum_{h=1}^HW_h\bar{X}_h$, where $W_h=\frac{N_h}{N}$ is called a weight function. Can someone explain intuitively what affect this weight function ...
0
votes
1answer
103 views

Data stratification for long-tailed distributions

I'm encountering a few problems successfully stratifying a dataset that is long-tailed to the right. In my research, I came across this excellent response: Sampling from a long tailed distribution ...
1
vote
1answer
709 views

Post stratification weights and the survey package in R

I have survey data that needs to be weighted, and to help me with this task, I have access to the full joint distributions of the variables I want to use. As I understand it, I should use the ...
3
votes
0answers
47 views

Can unintentional sample stratification be problematic?

An example of what I mean: I have a certain essay from all students at a university. I take a 1% random sample (not stratified) and run some time consuming computational analysis on each essay ...
2
votes
3answers
653 views

Calculating Sample Size for Non-Inferiority Test in Cox Proportional Hazards Regression With Stratification

I have a binary predictor, X, in a Cox proportional hazards regression model, and I want to show that it is NOT a significant predictor. In other words, I want to show that there is not a significant ...
0
votes
0answers
36 views

Ideas for sampling design

I have a data base of schools per district and the number of teachers per school. I want to select a sample where between 1 an 10 schools are selected in each district. Once a school is selected, the ...
3
votes
1answer
3k views

How to conduct Cox PH stratisified by matched pair in R

I am wanting to calculate hazard ratio in a matched cohort design. For such I am using the coxph() function in R. But I have been recommended to stratify by matched ...
7
votes
1answer
1k views

Is stratified meta-analysis more or less objective than meta-regression?

Reviewer asked me why I use meta-regression as a way how to deal with heterogeneity among effect sizes instead of conducting stratified meta-analysis. I tried to google "stratified meta-analysis" and ...
0
votes
1answer
519 views

Ignore strata in external validation of stratified Cox prop hazards model?

I've fit a stratified Cox proportional hazards model to some survival data, where I've stratified by a potential confounder which is the batch the data comes from (there are three batches). Now, I'd ...
1
vote
1answer
614 views

Deriving the formula for the mean of a stratified sample

The formula for the mean of a stratified sample $\bar Y_s$ is: $$\bar Y_s = \frac 1 N \sum_i N_i \bar Y_i$$ where $N$ is the sample size for all strata, and $N_i$ and $Y_i$ are the sample size and ...
28
votes
2answers
16k views

Why use stratified cross validation? Why does this not damage variance related benefit?

I've been told that is beneficial to use stratified cross validation especially when response classes are unbalanced. If one purpose of cross-validation is to help account for the randomness of our ...
1
vote
1answer
57 views

which statistical sampling technique to employ to have a representative sample

There is kwashiorkor outbreak in my community Abeng (a city in Africa). The community is in sections A to F and it heterogeneous in nature.
2
votes
2answers
413 views

How to partition a training-set when I have a big class imbalance?

In my actual data class A has 90%, class B has 9% and class C has 1% (numbers are made up for sake of simplicity). Now I want to prepare a training set for my classifier (I plan to use Vowpal Wabbit). ...
0
votes
2answers
1k views

Stratified or multi-stage sampling?

I just started volunteering with a small organization that wants to estimate the average and total number environmental trouble spots in a large US city. They have a clear definition of an ...
0
votes
0answers
96 views

How would I factor study design into relative risk?

I'm comparing various conditions versus some disease for a large stratified/clustered dataset which purports to account for the entire population after weighting -- hence the use of RR instead of OR. ...
1
vote
0answers
374 views

Stratified random sampling and its distribution

I'm learning stratified random sampling and I'm having hard time to solve this problem. consider the following population of six units: ...
1
vote
1answer
2k views

Individual and overall RMSE for multivariate data

I have a dataset which contains missing values, and I'm using imputation packages (Rs mi and ...
5
votes
1answer
2k views

Confidence interval for a proportion estimated through stratified sampling

When estimating the confidence interval for a binomial proportion $p$, with $\hat{p}$ near 0 or 1, one has to use something other than the Wald interval to get a reasonable estimate (see for instance ...
7
votes
1answer
2k views

Machine learning with weighted / complex survey data

I have worked a lot with various nationally representative data. These data sources have a complex survey design, so the analysis requires the specification of stratification and weight variables. ...
4
votes
0answers
635 views

Stratified Cross-Validation with Collaborative Filtering

My dataset consists of binary preferences ($0$ or $1$) given by users on items like this: User-ID | Item-ID | Preference If a user has not given a preference to an item, then it is not in the ...
1
vote
0answers
44 views

Accounting for noncoverage in survey

[I'm a stats noob, so please correct if I am going wrong somewhere!] We run a survey of our customers based on certain events or touchpoints. The responses vary significantly by these touchpoints, so ...