# How to calculate effect size of moderation analyses on repeated-measures data?

I conducted a moderation analysis on repeated-measures data using the MEMORE macro for SPSS (https://www.akmontoya.com/spss-and-sas-macros). However, I need standardized effect sizes but I haven't managed to figure it out and it's quite urgent.

So each participant read 2 character descriptions about a healthy (Condition C1) and unhealthy (Condition C2) male (independent variable) and had to judge the likability (outcome). They also had to score their gender system justification beliefs once (moderator).
MEMORE recalculates the outcome by taking a difference score of likability_C1 - likability_C2 at various levels of the moderator and then calculates a t-statistic to check significance. I got this output, with a mean-centered moderator:

Conditional Effect of 'X' on Y at values of moderator(s)

SystemX     Effect         SE          t          p       LLCI       ULCI
-1,2917      ,7257      ,0877     8,2786      ,0000      ,5535      ,8980
,0000      ,4285      ,0620     6,9172      ,0000      ,3068      ,5503
1,2917      ,1314      ,0877     1,4984      ,1347     -,0409      ,3036


How do I now get effect sizes?

• Why do you need "standardized effect sizes"? See this page and this page, for example; they don't add much in complex designs.There are many definitions of "effect size."; which do you want? For Cohen's d you need some estimate of a standard deviation (SD). Does MEMOIRE handle repeated measures with a mixed model, or some other way? As the above links indicate, with a mixed model there's both within-individual and among-individual SDs to consider.
– EdM
Apr 14, 2023 at 9:48
• Hi, thank you very much for replying to my question. I was told I need standardized conditional effects to be able to say whether my unstandardized conditional effects are large or not as my scales are arbitrary. I think I have found a solution here: Bodner, T. E. (2017). Standardized Effect Sizes for Moderated Conditional Fixed Effects with Continuous Moderator Variables. Frontiers in Psychology, 8. frontiersin.org/articles/10.3389/fpsyg.2017.00562 -- it takes the group mean difference divided by the square root of the mean sqaured error (MSE) Apr 14, 2023 at 13:22

The "effect size" you have in mind is the one discussed on this page, expressed in general as:

$$\text{Effect Size} = \dfrac{\text{Difference of Means}}{\text{Standard Deviation}}\text{.}$$

The values you show for Effect are presumably what you want to include in the numerator. The question then is what to use for the denominator, the Standard Deviation (SD).

One possibility, which you note in a comment, is to use the root-mean-square error from the model. That estimates the SD of a single "likability" score and is a typical choice in a simple model for the difference in means between independent groups.

From your description, it seems that your Effect values might be more related to paired comparisons within individuals. In that case, you probably should be using an SD estimate related to the paired differences. If that's the case and the pairing helped, you could be better off getting the SD estimates from the test results that you show, which would be related to the distribution of those paired differences.

As this answer shows, you can translate a standard error (SE) to an SD estimate if you multiply the SE by the square root of the number of observations. From the last row of your report, the combination of t-statistic and p-value suggests that you have about 470 observations for that row. Somewhere in the model there presumably is information about the degrees of freedom used for the t-tests reported with p-values of 0.

I'm not familiar with MEMOIRE, so it's possible that there's some additional subtlety here (like adjusting p-values for multiple comparisons). Also, remember that this type of effect-size calculation ignores the error in the SD estimate itself, which can be substantial.

Two final cautions. First, if repeated measures are handled by random effects in a mixed model, then there is the issue of how to combine the within-individual and among-individual SDs for use in the denominator above. If that's how MEMOIRE treats the repeated measures, then you need to take that into account. Second, if you are using a generalized linear model, then you want a different type of "effect size." See the links from this page, and the extensive discussion on this page.