Calculating effect size for glmm with repeated measures, what is the sample size?

Question

I am new to effect sizes and trying to calculate it for a repeated measure GLMM that looks like this:

variable ~ treatment * sampling occasion * year + (1|subject)

The variable is continuous, the treatment is categorical (two independent groups), the sampling occasion is discrete, the year is discrete, and subject is used as a random effect. The variable is measured at each sampling occasion on the same subject, hence the repeated measures.

Following Nakagawa & Cuthill 2007, I believe I should use the equation from Rosenthal (1994):

$$ d = t_{unpaired} + \sqrt{\dfrac{(n_1+n_2)}{n_1n_2}} $$

In this case, should I use the total n for each repeated measure or the real number of subjects in my experiment?

For example if I have 10 subjects in my first group, and 3 sampling occasion, is my n1 value 30 or 10?

Nakagawa, S., & Cuthill, I. C. (2007). Effect size, confidence interval and statistical significance: a practical guide for biologists. Biological reviews, 82(4), 591-605.

Answer 1

First, I think if you're interested in the point estimate for effect size, you should focus on the model estimate for the treatment effect. If that is big, your effect size is big. If it is small, your effect size is small. If it's unclear if it's big or not, it's hard to know...

If you need to standardize the effect (e.g. to compare to other effect sizes with different response scales and interpretations), I think Cohen's d (the first option in the paper you cite) is reasonable, but the t-based option you have above seems to be more about precision/confidence/standard error/significance than it is about how big of an effect you see (that measure always gets bigger with sample size, as the standard error of the mean goes down).

GLMMs are good for estimating effect sizes, but parametric hypothesis testing (and interval estimates) can be tough with them because figuring out the degrees of freedom is really tricky with random effects.