Relationship between Cohen's D effect size and p-value

Question

If a the Cohen's D of a study is 1.13, what would p = 0.003 and p = 0.05 Small, medium or large effect? I believe its a small effect but would like other thoughts and answers.

Answer 1

A p-value won't inform you about the importance of the effect size (here Cohen's d). As for interpreting p-values, you may find the following discussions relevant: Understanding p-values using an example and Interpretation of p-value near alpha level. There are many other similar threads on this website that can be useful to you.

As for the effect size (here, Cohen's d), judging if it's a small effect size depends on the context of the study. 1.13 may be an important effect in a given study, and a negligible effect in another study. It's a bit like asking "Is a weight loss of 5 grams a lot for an animal?" – it depends, are we talking about mice or elephants?

So you need to think about whether observing such an effect size matters or not. For instance, if you're testing the effect of some intervention, and observe this effect size, would the benefits related to this effect size justify generalizing this intervention to the whole population, if you consider the intervention's costs and side effects?

You may find out that there are some rules of thumbs meant to put labels such as "small", "medium" or "large" on effect sizes. For instance, in his book "Statistical Power Analysis for the Behavioral Sciences" (1988), Jacob Cohen uses the label "small" when $d = 0.2$, "medium" when $d = 0.5$, and large when $d = 0.8$. However, this kind of rule of thumb have been heavily criticized, including by Jacob Cohen, for the reasons mentioned above.

Cohen created these rules of thumbs for situations where it's impossible to use context to determine the importance of an effect, and recommended to prioritize using context whenever it's possible. However, one could argue the following points:

• Are there really situations where you absolutely can't use context to judge the importance of an effect size?
• If you really can't use context to judge if an effect is interesting or not, an alternative approach is to simply report the effect size value without attaching any qualitative adjective to it.

---

Edit following the discussion in comments:

When I mention taking into account costs and side effects, I do not necessarily mean in financial or health terms.

For example, a given intervention may be harmless to participants and costs financially almost nothing to implement in the population of interest. However, if the study that justified extending the intervention to the population is found out to be deeply flawed due to a poor design and its results to be wrong, it can undermine the trust that the population may have in future studies. You could consider that as a cost, so the study design is part of the context to take into account to judge the relevance of an effect size.