3
$\begingroup$

I am doing a study to find the correlation between onset-age-of-diabetes and BMI Z score. Our study found these two variables negatively correlated. This association is a bit controversial in literature as some of the studies found it positively correlated, others negatively correlated and no correlation at all.
We assume the lack of consistency between studies can be due to experimental designs or power of the study etc. I collected the old literatures and doing a post hoc test for calculating achieved power of these studies using G*Power software. My question is "Does it make sense to do a post hoc power analysis for study which has found the correlation between two variables as r = 0.01; p =0.82, as it already statistically insignificant.

$\endgroup$

1 Answer 1

2
$\begingroup$

It depends.

It makes sense to do a power calculation as to whether studies were powered for plausible effect sizes. If they were not, then it would be no surprise, if studies fail to show an effect and in comination with publication bias (and flexibility in analysis choices) might well mean that any results with some $p\leq 0.05$ are spurious. On the other hand, if they were well powered, then perhaps there is no effect of at least the hypothesized size.

What usually does not makes sense is a power analysis assuming the observed effect size. As Senn put it, those seem to only ever be done if p>0.05 and thus, invaribly, find a power <50%. Then researchers claim that their favorite effect might exist, after all...

Regarding the specific analysis, does it make sense to correlate these variables? Is it not more logical to do a survival analysis of a cohort of at risk patients? You would only have a time of onset for those that actually developed diabetes and that presumably leads to all sorts of weird selection effects (like all those publication that produce a "acceleration of the glucose trajectory towards diabetes" by plotting glucose values relative to the time when the exact same glucose values first exceed a particular threshold).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.