I using simulated data sets to compare the ability to two different experimental designs (A and B) to detect a interaction between two variables (x and y) in determining the observed output (o). The results appear contradictory to me, and I think there must be an aspect of how lmer works that I'm not understanding that can explain this.
I am using lmer from the lmerTest package in r to fit the stimulated data to a model, and calculate the coefficient, SE of the coefficient, and p-value:
fit<-lmer(O~x+y+x*y+(1|subjectID),data) c<-summary(fit)$coefficients p<-c['x:yTRUE','Pr(>|t|)'] beta<-c['x:yTRUE','Estimate'] betaSE<-c['x:yTRUE','Std. Error']
When I generate ~500 replicates of simulated data for each trial design (A and B), and run each replicate through this analysis, I find:
Design A has higher power (a larger fraction of the replicates find a statistically significant effect) and lower mean
betaSE (as defined above), despite the fact that Design B has a similar magnitude of mean
beta, a significantly smaller standard deviation of the
beta (SD calculated across the replicates).
What should this be telling me about why Design B has lower power to detect a significant effect?
EDIT: Designs A and B are two different clinical trial designs where participants spend different amounts of time on open label drug and blinded drug vs placebo. The simulated data are created with a complex set of flexible assumptions about the relationships between 3 factors in a model of symptom burden over time as a function of time, expectancy and drug response. These assumptions are used to create a covariance matrix, which is then used by
mvrnorm to generate simulated factor-level data as per the specified covariance matrix. The factors are summed to create simulated data that represents what would actually be produced by a clinical trial. This data is what is analyzed using the above methods.