3
$\begingroup$

I am trying to conduct a sensitivity power analysis for a (2x2) ANOVA. The participants all received trials based on color or shape, and items could be either old or new. In G*Power I have the following information included into the analysis:

a err prob = 0.05
Power = .80
Total sample size = 15
Number of groups = 1
Number of measurements = 4
Corr among rep measures = 0.5
Nonsphericity correct = 1

With this I get an effect size $f = 0.3158294$. I am not entirely sure what to do with this information or how to interpret it? The ANOVA repeated measure with-between interaction sensitivity test actually tells you the total sample size you need for the observed effect size. But this is giving me the effect size f, and I am not sure what that is. Should I be comparing this to the partial eta square value I received for my main effects and interactions?

$\endgroup$
2
  • 1
    $\begingroup$ Sensitivity power analysis isn't the same as post-hoc power analysis. I'll refer you again to D. Lakens's online book; see the section 8.18 Plot a Sensitivity Power Analysis. I don't know GPower but came across this tutorial, Power analysis using G*Power, that seems to explain the concepts quite well. Good luck with getting your paper published. $\endgroup$
    – dipetkov
    Oct 11, 2023 at 12:56
  • $\begingroup$ PS: You didn't ask about this but your previous question makes it clear that you split your already small dataset (n = 45 subjects) to do 3 separate subset analyses with n = 15. This doesn't seem like a great approach. The full model can give you all the inferences that you need. You also did lots of tests, so you probably should be looking at some kind of multiplicity correction. $\endgroup$
    – dipetkov
    Oct 11, 2023 at 12:59

1 Answer 1

1
$\begingroup$

Post-Hoc Power Analysis

There is no point in performing a post-hoc power analysis (conducting power analyses after samples are obtained). See the citations below, but it basically boils down to 1) your sample size is already known and 2) you presumably also already know your effect size.

References

  • Dziak, J. J., Dierker, L. C., & Abar, B. (2020). The interpretation of statistical power after the data have been gathered. Current Psychology, 39(3), 870–877. https://doi.org/10.1007/s12144-018-0018-1
  • Lakens, D. (2022). Sample size justification. Collabra: Psychology, 8(1), 33267.
  • Zhang, Y., Hedo, R., Rivera, A., Rull, R., Richardson, S., & Tu, X. M. (2019). Post hoc power analysis: Is it an informative and meaningful analysis? General Psychiatry, 32(4), e100069. https://doi.org/10.1136/gpsych-2019-100069
$\endgroup$
5
  • $\begingroup$ I understand. But if that's what the reviewers are requiring, it's what I have to do. $\endgroup$
    – gert
    Oct 10, 2023 at 22:53
  • $\begingroup$ No it isn't. You send them those references and explain why their request is unreasonable. Else send your analysis elsewhere. $\endgroup$ Oct 10, 2023 at 22:55
  • $\begingroup$ Thanks for the references. But I am not in a position to do that, so I am going to continue to search for an answer to the question I have asked. $\endgroup$
    – gert
    Oct 10, 2023 at 22:58
  • 1
    $\begingroup$ I don't think it is very scientific to engage in this practice and I strongly suggest actually reading those references if you want to know why this is a silly request. It appears you have also ignored the warnings of another user in another post you made, so I don't know what other answers you are trying to seek if you have been told twice that this practice is poor. $\endgroup$ Oct 10, 2023 at 23:09
  • 1
    $\begingroup$ I have read the articles. I fully intend to discuss with my PI that we should reach out to the reviewers about why the sensitivity test they have suggested is not a good option. However, I cannot go over my PIs head and send the references to the reviewers if they do not want me to, it's not my place, it's not my grant. I would like to know how to do the sensitivity test regardless because from the reading I have done, they are helpful for other things. Again, thank you for your input, but I am simply looking for someone to explain to me how to interpret the test. $\endgroup$
    – gert
    Oct 10, 2023 at 23:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.