What is the new sample size of a within study summary effect for outcomes with different sample sizes?

Question

I am performing a meta-analysis and want to compute a summary effect (i.e. weighted mean) for studies that report data on more than one effect size. The same participants are involved, however the number of participants for each effect size may differ. To take into account the different sample sizes, the weight (W) assigned to each effect size (ES) would the inverse of that effect size’s variance (V), i.e.:

W = 1 / V

The weighted mean (M) of the combined effect sizes then would be:

M = ∑ ES x W / ∑ W

With variance of this new mean:

V = 1 / ∑ W

The summary effect (i.e. weighted mean) is used in the subsequent meta-analysis with the effect sizes (means) of other studies. To be able to do this, I need the sample size (N) for each effect size. My question therefore is: what would be the (new) sample size of a summary effect if the sample sizes for the effect sizes that together make up the summary effect are different, e.g. n=105, n=115, and n=120?

The method above assumes the combined effect sizes (within a study) are independent (r = 0). This is very unlikely. My second question, therefore, is: how can I incorporate the correlation between the effect sizes within a study and, at the same time, take into account the different sample sizes for these effect sizes (assigning weight to each effect size based on the inverse of that effect size’s variance)?

Thank you in advance for your help!

Answer 1

You cannot just treat $M$ as if it was an individual effect size with some new sample size. Here, you already have the sampling variance ($V$) for this (combined) estimate and that is what would go into the meta-analysis (the sample size would only be needed in order to compute the sampling variance, but you already have it).

In order to properly take the correlation between the effects into consideration, you would need to compute their covariances. See, for example, chapter 24 in Borenstein et al. (2009), Introduction to Meta-Analysis. Since the information needed to compute the covariances is often not available (specifically, the correlation between the variables underlying the different effect sizes that were computed), one could make a 'guestimate' about their correlation and then use that.

The equations for combining the effect sizes into a single aggregated effect size are as follows (using inverse-variance weighting): Let $V$ be the $k \times k$ var-cov matrix of the $k$ estimates in $y$. Then the combined effect size is $M = (X'WX)^{-1} X'Wy$, where $W = V^{-1}$ and $X$ is just a column vector of 1's. The sampling variance of $M$ is then $(X'WX)^{-1}$.