Question
How do I find the effect size for the different hierarchical multiple-level regressions used by papers in my review? And how do I categorise their effect size?
Detail
I’m publishing a systematic review of 22 psychology papers that use a range of statistical analyses and the reviewers have asked me to report the effect size of each study’s reported outcome (I’m assuming they mean with reference to the specific factors I’m reviewing).
This is easy for paper which use correlations and ANOVA. For those I’m using Cohen’s d (CV1, CV2) and categorising them into “small”, “medium”, “large” using Cohen’s rule of thumb (e.g. as the asker of this question did: CV3, also CV1 again)
How do I find the effect size for the different hierarchical multiple-level regressions used by papers in my review? And how do I categorise their effect size?
One option would seem to be CV4, but this seems to require a multiple regression model for all of the factors except the one I need to calculate, and this information isn't available in the papers. So I think the answer to my question depends on the statistics provided by the other papers…
(1) Petrowski et al. (2011) report Beta, p value and R^2 (Hierarchical Linear Model), and Tyrrell et al. (1999) report b, Beta, change in R^2 and dfs and p value (HLM). Without r for individual variables is there anything I can do to calculate the effect size for the single factor I care about?
(2) Sauer et al. (2003) report regression coefficients (not stating whether it is standardised or not), Standard Error, t statistic and r for fixed effects, and Variance Component and Standard Deviation, and X2 for random effects (HLM). Can I use the r for fixed effects here?
(3) Schauenburg et al. (2010) report regression coefficients (but don’t state whether it is standardised or not), Standard Error, p value and t statistic (Multilevel Regression). Can I calculate Cohen’s d or r from these values?
The full references of the four papers I mention here, in case they're helpful...
- Petrowski, K., Nowacki, K., Pokorny., D. & Buchheim, A. (2011). Matching the patient to the therapist. The roles of the attachment status and the helping alliance. The Journal of Nervous and Mental Disease, 199(11), 839-844. doi:10.1097/NMD.0b013e3182349cce
- Sauer, E. M., Lopez, F. G., & Gormley, B. (2003). Respective contributions of therapist and client adult attachment orientations to the development of the early working alliance: A preliminary growth modelling study. Psychotherapy Research, 13(3), 371–382. doi:10.1093/ptr/kpg027
- Schauenburg, H., Buchheim, A., Neckh, K., Notle, T., Brenk-Franz, K., Leichsenring, F., Strack, M., Dinger, U. (2010). The influence of psychodynamically oriented therapists’ attachment representations on outcomes and alliance in inpatient psychotherapy abstract. Psychotherapy Research, 20(2), 193-202. doi: 10.1080/10503300903204043
- Tyrrell, C. L., Dozier, M., Teague, G. B., & Fallot, R. D. (1999). Effective treatment relationships for persons with serious psychiatric disorders: The importance of attachment states of mind. Journal of Consulting and Clinical Psychology, 67(5), 725-733. doi:10.1037//0022-006X.67.5.725
