5
$\begingroup$

I'm trying to test a prediction from a computational model with a biological experiment.

The model predicts a linear dependence between cell size and asymmetry. I performed an experiment that is known to affect cell size, and got a nice significant slope when regressing asymmetry onto cell size.

However, in biology, there is rarely a perfectly clean experiment. To account for potential other effects of the perturbation, I added a dummy variable for the perturbation to the regression. The coefficient of the dummy is not significant (p=0.5), but the coefficient of cell size is no longer significant, either (p=0.1).

Assuming that the model prediction is correct, how can I, given the results of my regression, calculate how many additional measurements I need to perform so that the coefficient of cell size remains significant even in the presence of the confounding dummy?

$\endgroup$
1
$\begingroup$

Consider this: given no population effect, and given an initial sample effect classified as significant and a 2nd-stage effect (controlling for dummy variable) classified as non-significant, what is the probability that a researcher will continue to amass more observations to the point where the effect in question appears significant? I suspect it is very high, but that doesn't shed much light on your research question. The literature on Type I errors, researcher bias, and the File Drawer Problem will be relevant here. I realize this will not seem to directly answer your question, but hopefully it will reveal a problem with the question and will prompt additional thinking about the role of significance tests.

More directly to the point: you can treat the partial correlation coefficient you've obtained when controlling for the dummy as the best estimate and can then conduct a garden-variety power analysis to see what sample size would be needed for such a correlation coefficient to appear significantly different from zero. There are plenty of online correlation power calculators that will enable you to do this (e.g., at Vassar's site), or you can download the free program GPower. If you need a deeper understanding of the uncertaintly surrounding this power question, you could conduct a Monte Carlo simulation in which both your focal variable and the dummy have coefficients that can vary in proportion to their standard errors.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.