3
$\begingroup$

Let's say that I, for a meta-analysis, have two effect sizes, $ES_1$ and $ES_2$ (begin standardized mean differences) with standard errors, $SE_1$ and $SE_2$, and that I want to combine these into a single measure (they might for example represent two different outcomes on the same test) with a single standard error, $ES_{combined}$ and $SE_{combined}$. Further, I assume that they are fully correlated, meaning that the number of effect sizes that I combine should not affect how precise the final effect size is (that is, how narrow its standard error is).

I've found some sources claiming that you simply can compute the averages, that is: $ES_{combined}=\frac{ES_1+ES_2}{2}$ and $SE_{combined}=\frac{SE_1+SE_2}{2}$, but they have not supplied a rationale for this reasoning.

How can I do this correctly?

$\endgroup$
2
  • $\begingroup$ there is no rationale because if you assume something like full correlation, anything can be said about the "real" effect size, it depends on how you justify the difference between measured effect sizes, and it's totally subjective. you could state that the combined ES is the smaller of the two reported, or that it's the bigger one. where does this "full correlation" assumption come from? $\endgroup$
    – carlo
    Commented Apr 19, 2020 at 14:10
  • $\begingroup$ @carlo It's meant to be indicative of the fact that no measure should be weighted more strongly than any other when it comes to the computation of the final measure. $\endgroup$
    – Speldosa
    Commented Apr 19, 2020 at 15:26

1 Answer 1

1
$\begingroup$

Keep in mind that $ES_i$ are random variables, in your case $ES_{combine}$ is just a linear combination of random variables. $$ ES_{combine}=\sum_i^n w_i ES_i $$ Where $n=2,w=1/2$. So,

$$ \begin{align} mean(ES_{combine})=\sum_i^n w_i \ mean(ES_i) \\ sd(ES_{combine})=\sqrt{\sum_i^n w_i^2 \ var(ES_i) + \sum_{ij\in n} 2 w_i w_j cov(ES_i ES_j)} \end{align} $$

Also, its interesting to point out that by taking average you are assuming the effective sizes play the same importance. But in reality some information source might be more reliable than others. There are some truth finding models that can help you calculate the reliabilities of the information sources.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.