A while ago I posted a question regarding performing post-hoc power analyses in meta-analyses and was given a couple of useful links to discussions regarding why this isn’t a good idea. From my reading of these discussions, and the suggested academic articles, one should rather than trying to do a post-hoc power analysis look at the confidence interval of the effect size of a null result to discern whether it contains an effect size that one would deem as relevant.

This is all well and good when having just a single effect size that isn’t significantly refuting the null hypothesis. However, how should one approach power indicators for omnibus tests of several effect sizes? For example, I might want to do a moderator analysis with two or more levels of a moderator, yielding an estimate and confidence interval for each moderator level. How can I in this case talk about the sensitivity of my test?

  • 1
    $\begingroup$ You can look at each moderator separately. I suppose that if you wanted to you could generate a confidence ellipse around two estimates, and so on into hyperspace. $\endgroup$ – mdewey Nov 30 '16 at 18:22
  • $\begingroup$ @mdewey Thanks, I hadn't heard about confidence ellipses before. I'll check it out. $\endgroup$ – Speldosa Dec 6 '16 at 13:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.