2
$\begingroup$

I am conducting a meta-analysis on the effect of conscientiousness on scholastic achievement in high-school students. Currently, I am struggling to find the right statistical analysis for my data structure. Several studies are reporting multiple (dependent) effect sizes. When there are multiple effect sizes that are based on the same sample, these effect sizes are dependent. One way to account for this dependency is a three-level meta-analysis. The hierarchical data structure would be as follows:

  • L-1: study participants (are nested within effect sizes)
  • L-2: effect sizes (are nested within studies)
  • L-3: studies

enter image description here

As I am quite confident in handling this kind of dependency using three-level meta-analysis, I struggling with another kind of dependency in my data.

Some studies report effect sizes of multiple (independent) samples within one study. The effect sizes are now independent in the sense that they are based on different participants. However, all samples were tested by the same research team, using the same study design, and sometimes even in the same school/class (e.g. effect sizes are reported for boys and girls separately). Is it possible to account for this kind of dependency in a three-level meta-analysis?

Best regards,

$\endgroup$
1
  • $\begingroup$ It looks like you have a fourth level which introduces computational complexity but in theory should work. $\endgroup$
    – mdewey
    Dec 7, 2018 at 15:43

1 Answer 1

0
$\begingroup$

In the example you have given where effect sizes are reported for boys and girs separately within a single study, this is a type of hierarchical effects. Another type of hierarchical effects (i.e., the typical one) is that research group conducts multiple studies based on independent samples and each study reports one effect estimate. If I understand correctly, the two types can be treated as equivalent and modelled in the three-level structure.

Recently, a new approach has been proposed: the Correlated and Hierarchical effects model, which combines the multi-level model with Robust variance estimation method. I am using it in my project.

Reference: Meta-analysis with Robust Variance Estimation: Expanding the Range of Working Models https://link.springer.com/article/10.1007/s11121-021-01246-3

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.