I am very confused if Cohen's $f^2$ in ANOVA is $\frac{MSTr}{MSE}$ or $\frac{SSTr}{SSE}$ where:

  • $SSTr$: Treatment Sum of Square
  • $SSE$: Error Sum of Square
  • $MSTr$: Mean SSTr
  • $MSE$: Mean SSE

$MSTr = SSTr/(I-1)$ and $MSE = SSE/(n - I)$, where $I$ is the number of groups and $n$ is the total number of observations.

Some information from the Internet says Cohen's $f^2$ is defined as $R^2/(1-R^2)$ for both regression and ANOVA where $R^2=SSTr/SST$. In that case $f^2=SSTr/SSE$ but other sources seems to state that it is $\frac{MSTr}{MSE}$ in ANOVA.

  • $\begingroup$ Tip: How would you express MST and MSE in terms of SST and SSE? And how would you express, as a consequence, the ratio MST/MSE in terms of SST/SSE? $\endgroup$ – Sextus Empiricus Feb 4 at 7:48
  • $\begingroup$ Criticism: Explanations like "SSTr: SS Treatment" are a bit cryptical. What do you mean exactly by 'SS Treatment'? $\endgroup$ – Sextus Empiricus Feb 4 at 7:50
  • $\begingroup$ @SextusEmpiricus, I updated the question. $\endgroup$ – Royalblue Feb 4 at 7:55
  • $\begingroup$ How many squares do you sum to get SSTr (treatment sum of square)? Could you express it as a formula? $\endgroup$ – Sextus Empiricus Feb 4 at 8:02
  • $\begingroup$ If we denote $J$ as the number of observations in each group, that will be $IJ$ squares. $\endgroup$ – Royalblue Feb 4 at 8:17

A useful reference is "Statistical power analysis for the behavioral sciences" by Jacob Cohen. Chapter 8, about ANOVA, is available via google https://books.google.ch/books?id=rEe0BQAAQBAJ&&pg=PA273 or other online sources.

The value for Cohen's f resembles Cohen's d

  • Cohen's d is the ratio of the difference between two population means and the population variance:

$$d = \frac{\mu_1 - \mu_2}{\sigma}$$

  • Cohen's f generalizes to more than two populations and is the ratio of the difference between the population means (in terms of their variance) and the variance/deviation of the populations.

$$ f = \frac{\sigma_ \mu}{\sigma}$$

You can relate these variances roughly with the sum of squares.

$$\sigma_\mu^2 \approx SS_{between}/n$$


$$\sigma^2 \approx SS_{within}/n$$

Such that

$$f^2 \approx \frac{SS_{between}}{SS_{within}}$$

You do get some discrepancies because the computation of an average square (using the denominator $n$) is not an unbiased estimator for the variances $\sigma_\mu$ and $\sigma$, and instead you could use some correction terms (which becomes complex for unequal groups) but they will not have a large effect.

F-statistic and F-test

You do have a related term the F-statistic and this is computed as

$$F = \frac{SS_{between}/(I-1)}{SS_{within}/(n-I)}$$

where $n$ is the number of observations and $I$ the number of groups.

This is not the same as Cohen's f.

The numerator $SS_{between}/(I-1)$ relates to the random observed sample variance for the group means given a null hypothesis where the effect is null (absent), $\sigma_\mu = 0$. In that case the error in the group means is solely due to a random effect and is distributed with a variance that decreases as $n$ is larger.

Overall you will have that, given the null hypothesis $\sigma_\mu = 0$ is true, this between error is distributed as a chi-squared variable $SS_{between} \sim \chi^2_{I-1}$ independent of the sample size $n$ (as $n$ grows, the increase in the number of term/squares that are summed will cancel with the decreasing size of those terms/squares).

However, in case there is an effect $\sigma_\mu \neq 0$, then the $SS_{between}$ will grow roughly linear with $n$ and so you have to divide by $n$ and use the average $SS_{between}$. (it will follow approximately a noncentral chi-squared distribution and exactly when the populations are normal distributed)

  • $\begingroup$ Clear and definitive. Thank you for your great teaching! Indeed, as we evaluate a statistic under the assumption of a null hypothesis in order to test(reject) the null hypothesis, it make sense that we should evaluate the statistic under a alternative hypothesis in order to measure the effect size of the alternative. $\endgroup$ – Royalblue Feb 5 at 0:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.