I am working with a complex sample survey data using Stata to determine the predictors of a given dependent variable. As part of my analysis result I want to report the post-hoc power for each predictor. However, post-hoc power analysis in Stata (and other softwares I know) is done assuming the data is generated through a simple random sampling. I thought dividing the complex survey sample size by the design effect used in estimating the sample size to get the effective sample size and then doing the power analysis on the effective sample size. However, I am not sure if that is a proper course of action. What would you suggest in this regard?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ The answer to this post on CV addresses power analysis for ordinal logistic regression using simulation method in R. May it be possible to do simulation for post-hoc power analysis for complex sample linear regression that involves multiple imputation (preferably in Stata)? $\endgroup$– Ayalew A.Commented Jan 29, 2015 at 14:53
-
$\begingroup$ Another post on CV addresses power analysis with weighted survey data. The answers suggest using simulation to estimate power. However, how to do simulation for complex sample surveys (involving stratification, clustering and weighting) is not shown in the answers. In fact, my data is also complicated by multiple imputation. $\endgroup$– Ayalew A.Commented Jan 30, 2015 at 8:13
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
I did exactly this in a recent secondary data analysis of complex survey data. I was performing a two-level mixed regression model and ascertained the power of the study to detect significant differences given different effect size estimates (0.2, 0.5, 0.8).
I used three different ICC values (based on previous research) to estimate three estimate design effects. Used these to determine three candidate effective sample sizes. And then determined the power of the study (for each predictor) to detect small, medium, and large effect size differences.
