Analysing stratified random sample Let's say I am analysing an experimental design with treatment and control group which was based on random sampling design stratified on a number of covariates. When analysing the data I see two options:


*

*include the stratification variables as covariates

*set a complex survey design option that accounts for stratification


Option 1 is the option I see done in academic papers in my literature, but option 2 makes more sense to me. I am wondering whether to two procedures are equivalent or if option 2 gains more power?
 A: Some thoughts


*

*An important issue in this context is a correct estimation of standard errors, and therefore p-values, confidence intervals and thus overall levels of statistical significance of your independent variables. Being more specific, the accuracy of standard errors in complex sample designs is commonly compared against a Simple Random Sample (SRS). So compared to the SRS, standard errors under stratification design yield smaller standard errors. Now, since you have a Stratified Random Sample which implies a different probability of selection + stratification (in contrast to the SRS where the probability of selection is the same for all sampled units), I think it is generally difficult to speculate what standard errors will be like compared to the SRS.

*However, I would think about the problem this way. Assuming stratification by some variables is indeed an important part of your design, calculating stratified standard errors will give you more confidence in the accuracy of your standard errors. Additionally, assuming that your stratified variables are for example gender or age group means that you have that data also recorded. 

*Now you may do the following as one way of going about this problem. You can indeed set a complex survey design with stratification. Then, if you have doubts about your stratification results, you can examine how your model is affected, if at all, by the addition of stratification covariates (really coded as dummy variables in this sense). 

*A cautionary note: assuming you also want to include a number of stratification variables as dummy covariates, please be aware that the inclusion of too many of such variables will make you capitalise on chance that some of the variables will be statistically significant by chance. For example, if you have 5 IVs and you decide to include 15 more dummy-coded covariates, this will total 20 IVs in your model, meaning that on average 1 of these variables is going to be statistically significant by chance. Hopefully, this might not apply to you, but it's good to be aware of this.   
