Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
What is the theoretical justification for using the square root of the sample size as the weight when combining z-scores in a meta-analysis?
Is this because the variance of the z-score is proportional to 1/n, where n is the sample size, so the inverse variance is proportional to n?
Let's look at the CLT. Its main issue is convergence in distribution. Rather than writing $Z=\frac{\bar{X}-\mu}{\sigma/\sqrt{n}}\sim\mathcal{N}(0,1)$, we can write $\frac{\bar{X}-\mu}{\sigma/\sqrt{n}}\xrightarrow{d}\mathcal{N}(0,1)$. The CLT has some variants, the Lindeberg–Lévy variant states it in a slightly different manner: $\sqrt{n}(\bar{X}-\mu)\xrightarrow{d}\mathcal{N}(0,\sigma^2)$. This means that the difference between the average and the mean converges to the normal distribution $\mathcal{N}(0,\sigma^2)$ with rate $\sqrt{n}$.
For our needs, we can re-write the Lindeberg–Lévy variant as $\sqrt{n}\left(\frac{\bar{X}-\mu}{\sigma}\right)\xrightarrow{d}\mathcal{N}(0,1)$. That is, the statistic $\frac{\bar{X}-\mu}{\sigma}$ converges to the standard normal distribution $\mathcal{N}(0,1)$ with rate $\sqrt{n}$.
Why is this important? One can think of the rate of convergence as the speed of approaching $\mu$. Upon combining different variables (which depends on some theory like continuity conditions, Slutsky's theorem and the LLN), we need to make sure they have the same rate of convergence or otherwise the convergence of the sum doesn't hold. Consider $Z_1=\frac{\bar{X}-\mu_X}{\sigma_X}$ and $Z_2=\frac{\bar{Y}-\mu_Y}{\sigma_Y}$. If they have the same convergence rate (say $\sqrt{n}$) then we can write something like $\sqrt{n}(Z_1+Z_2)\xrightarrow{d}\mathcal{N}(0,2)$. If they have different convergence rates, we cannot discuss the convergence of a combination.
This issue of rate of convergence (which in CLT is $\sqrt{n}$) is the reason for writing the square root of the sample size in z-scores. The rate of convergence is an important and nontrivial topic, you can read some more here and here.
Z-scores are estimates scaled by the estimated standard error of those estimates. Say we have two estimates from two samples $$Z_1 = \frac{\bar{X}_1}{\sigma\sqrt{n_1}} = \frac{1}{\sigma\sqrt{n_1}} \sum_{k=1}^{n_1} X_{1k} \\ Z_2 = \frac{\bar{X}_2}{\sigma\sqrt{n_2}} = \frac{1}{\sigma\sqrt{n_2}} \sum_{k=1}^{n_2} X_{2k} $$
where $\sigma$ is the error in the individual observations $X_{ik}$ (which for simplicity we assume as the same in both groups) and $n_1$ and $n_2$ are the sample sizes.
Here you see that if you would simply add $Z_1$ and $Z_2$ together with equal weights then the individual observations $X_{ik}$ do not get equal weights which is less efficient.
$$\sqrt{0.5} Z_1 + \sqrt{0.5} Z_1 = \sum_{i \in [1,2]} \sum_{k=1}^{n_i} \left( \frac{\sqrt{0.5}}{\sigma\sqrt{n_i}} X_{ik} \right)$$
The justification is that a weighted mean will have a smaller standard error, in comparison to an unweighted arithmetic mean. (From a different perspective the justification is a larger mean z-score instead of smaller standard error, see at the end)
This relates a bit to generalized least squares, and weighted least squares, which have the task to compute an estimate when the observed data points do not have equal variance/error.
What you are doing is computing an average which is a linear estimator and generalized least squares, which will use a weighted mean based on the variance of the individual terms, is the best linear unbiased estimator.
Example of a weighted mean having lower variance:
Example: If you have two observations distributed as
$$\bar{x}_1 \sim N(\mu, \sigma_1^2)\\ \bar{x}_2 \sim N(\mu, \sigma_2^2)$$
then the weighted mean (with weights $a_1+a_2=1$) will be distributed as
$$a_1 z_1 +a_2 z_2 \sim N(\mu, \sigma^2)$$
with the variance a weighted sum $$\sigma^2 = a_1^2 \sigma_1^2 + a_2^2 \sigma_2^2 = \sigma_1^2 -2 a_2 \sigma_1^2 + a_2^2 (\sigma_2^2+\sigma_1^2)$$
which has a minimum in $a_2 = \frac{\sigma_1^2}{\sigma_1^2+\sigma_2^2}$.
Intuitively:
Imagine you have a study with sample size 1000 and a study with sample size 10. The estimate from the latter study has a very large error. When you would average the results of both fifty-fifty then the propagation of error from the inaccurate study will count fifty percent and will lead to a large error in the end result.
The unweighted average of two numbers, one with small error and one with large error, will not have a small error but a medium error. So the average makes the situation worse (because you already had a number with a small error).
Is this because the variance of the z-score is proportional to 1/n, where n is the sample size, so the inverse variance is proportional to n?
The z-scores have variance 1, because they are normalised, but they have different means.
The z-score is roughly distributed as $N(\mu/\sqrt{n},1)$. Where $\mu$ is the population mean. If $\mu$ is non-zero then larger samples will have larger z-scores and that is why you want to give them a stronger weight.
So the comparison with GLS above, which is about different variance instead of different means is a bit twisted, but superficially the principle is related. If you compute the z-scores back to means of the population, then the GLS comparison counts and the goal is to get a linear sum that estimates the population mean and has the lowest variance possible (smaller variance means a larger z-score).
