0
$\begingroup$

If a the Cohen's D of a study is 1.13, what would p = 0.003 and p = 0.05 Small, medium or large effect? I believe its a small effect but would like other thoughts and answers.

$\endgroup$
1
  • 4
    $\begingroup$ A p-value doesn't inform you about the magnitude of an effect. Cohen's d is a standardized effect size measure, and a d of 1.13 is typically considered a large effect, but of course it needs to be interpreted within the particular research context. In concrete terms, it would mean that the mean difference between the two groups is larger than one standard deviation of the outcome measure. $\endgroup$
    – Sointu
    Nov 8, 2023 at 9:04

1 Answer 1

5
$\begingroup$

A p-value won't inform you about the importance of the effect size (here Cohen's d). As for interpreting p-values, you may find the following discussions relevant: Understanding p-values using an example and Interpretation of p-value near alpha level. There are many other similar threads on this website that can be useful to you.

As for the effect size (here, Cohen's d), judging if it's a small effect size depends on the context of the study. 1.13 may be an important effect in a given study, and a negligible effect in another study. It's a bit like asking "Is a weight loss of 5 grams a lot for an animal?" – it depends, are we talking about mice or elephants?

So you need to think about whether observing such an effect size matters or not. For instance, if you're testing the effect of some intervention, and observe this effect size, would the benefits related to this effect size justify generalizing this intervention to the whole population, if you consider the intervention's costs and side effects?

You may find out that there are some rules of thumbs meant to put labels such as "small", "medium" or "large" on effect sizes. For instance, in his book "Statistical Power Analysis for the Behavioral Sciences" (1988), Jacob Cohen uses the label "small" when $d = 0.2$, "medium" when $d = 0.5$, and large when $d = 0.8$. However, this kind of rule of thumb have been heavily criticized, including by Jacob Cohen, for the reasons mentioned above.

Cohen created these rules of thumbs for situations where it's impossible to use context to determine the importance of an effect, and recommended to prioritize using context whenever it's possible. However, one could argue the following points:

  • Are there really situations where you absolutely can't use context to judge the importance of an effect size?
  • If you really can't use context to judge if an effect is interesting or not, an alternative approach is to simply report the effect size value without attaching any qualitative adjective to it.

Edit following the discussion in comments:

When I mention taking into account costs and side effects, I do not necessarily mean in financial or health terms.

For example, a given intervention may be harmless to participants and costs financially almost nothing to implement in the population of interest. However, if the study that justified extending the intervention to the population is found out to be deeply flawed due to a poor design and its results to be wrong, it can undermine the trust that the population may have in future studies. You could consider that as a cost, so the study design is part of the context to take into account to judge the relevance of an effect size.

$\endgroup$
2
  • $\begingroup$ Thank you everyone for your input. I appreciate this. For reference, this is the article. Bai, K. Y., Liu, G. H., Fan, C. H., Kuo, L. T., Hsu, W. H., Yu, P. A., & Chen, C. L. (2023). 12-week curcumin supplementation may relieve postexercise muscle fatigue in adolescent athletes. Frontiers in nutrition, 9, 1078108. $\endgroup$
    – user400184
    Nov 8, 2023 at 12:03
  • $\begingroup$ @user400184 To assess the results from this article, you might need some input from a researcher specialized on sport and nutrition. Now, besides effect size, some things I'd question about this article (based on a quick reading of the abstract, so take my opinion with a grain of salt) is the small number of participants (for instance, it can affect effect size by possibly exaggerating it; for more details see Button et al., 2013: nature.com/articles/nrn3475). The absence of randomization and the apparent absence of using a placebo in the control group may be also a problem. $\endgroup$
    – J-J-J
    Nov 8, 2023 at 20:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.