I did an experiment and afterward conducted an ANOVA where I got results that were not significant:

            Df Sum Sq Mean Sq F value Pr(>F)  
type         1   1.91   1.910   0.735 0.3951  
sex          1   7.48   7.482   2.878 0.0955 .
type:sex     1   0.05   0.048   0.018 0.8927  
Residuals   54 140.37   2.599 

I'm mostly interested in the type*sex interaction here.

My sample size is 16, pretty similar to other studies in this field (sample size usually 20 more or less). I've read that it's pointless to do a power analysis after the fact so now I'm wondering what sort of analysis I can do to test if my sample size was large enough to detect an effect.

When looking at the graphs they are in the expected direction for this variable and also for two other (out of 3) variables however they all similarly have very high p-values.

  • 2
    $\begingroup$ In addition to the answer from @rvl, a well respected expert on this issue, note that the main effects in your model dwarf the interaction term of interest to you. Even if an extremely large study showed that this interaction was "statistically significant," would it really be practically significant? $\endgroup$ – EdM Mar 23 '18 at 1:23

First, fit the best model you can to the data. Do all the appropriate diagnostic tests to check that assumptions are not violated, and that there are no observed variables that could have helped explain unexplained variation without overfitting.

Then, the test of whether your sample size was large enough to detect an effect is to simply observe whether or not you detected an effect. If you did, you had enough data to do so. If you didn’t, then you didn’t have enough data to do so.

It is no more complex than that, and those who try to make it so are just creating confusion.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.