4
$\begingroup$

What is the theoretical justification for using the square root of the sample size as the weight when combining z-scores in a meta-analysis?

Is this because the variance of the z-score is proportional to 1/n, where n is the sample size, so the inverse variance is proportional to n?

$\endgroup$

2 Answers 2

1
+50
$\begingroup$

Let's look at the CLT. Its main issue is convergence in distribution. Rather than writing $Z=\frac{\bar{X}-\mu}{\sigma/\sqrt{n}}\sim\mathcal{N}(0,1)$, we can write $\frac{\bar{X}-\mu}{\sigma/\sqrt{n}}\xrightarrow{d}\mathcal{N}(0,1)$. The CLT has some variants, the Lindeberg–Lévy variant states it in a slightly different manner: $\sqrt{n}(\bar{X}-\mu)\xrightarrow{d}\mathcal{N}(0,\sigma^2)$. This means that the difference between the average and the mean converges to the normal distribution $\mathcal{N}(0,\sigma^2)$ with rate $\sqrt{n}$.

For our needs, we can re-write the Lindeberg–Lévy variant as $\sqrt{n}\left(\frac{\bar{X}-\mu}{\sigma}\right)\xrightarrow{d}\mathcal{N}(0,1)$. That is, the statistic $\frac{\bar{X}-\mu}{\sigma}$ converges to the standard normal distribution $\mathcal{N}(0,1)$ with rate $\sqrt{n}$.

Why is this important? One can think of the rate of convergence as the speed of approaching $\mu$. Upon combining different variables (which depends on some theory like continuity conditions, Slutsky's theorem and the LLN), we need to make sure they have the same rate of convergence or otherwise the convergence of the sum doesn't hold. Consider $Z_1=\frac{\bar{X}-\mu_X}{\sigma_X}$ and $Z_2=\frac{\bar{Y}-\mu_Y}{\sigma_Y}$. If they have the same convergence rate (say $\sqrt{n}$) then we can write something like $\sqrt{n}(Z_1+Z_2)\xrightarrow{d}\mathcal{N}(0,2)$. If they have different convergence rates, we cannot discuss the convergence of a combination.

This issue of rate of convergence (which in CLT is $\sqrt{n}$) is the reason for writing the square root of the sample size in z-scores. The rate of convergence is an important and nontrivial topic, you can read some more here and here.

$\endgroup$
5
  • $\begingroup$ What does the letter d means? $\endgroup$
    – John Smith
    Mar 28 at 4:33
  • $\begingroup$ "If they have different convergence rates, we cannot discuss the convergence of a combination." Convergence rate = sqrt(n). Do you mean if there if a different sample size, we cannot discuss the convergence of a combination? But in meta analysis, sample size of individual studies often differs. They will combine by sqrt(n). $\endgroup$
    – John Smith
    Mar 28 at 6:01
  • 1
    $\begingroup$ The letter d means convergence in distribution $\endgroup$
    – Spätzle
    Mar 28 at 6:56
  • $\begingroup$ different sample sizes don't mean they can't converge. $\sqrt{n_1}$ and $\sqrt{n_2}$ are both rates of $\sqrt{n}$. The problem is, for example, when one converges with rate $\sqrt{n}$ and the other with rate $n$. $\endgroup$
    – Spätzle
    Mar 28 at 6:58
  • $\begingroup$ This has nothing to do with the CLT. $\endgroup$ Mar 28 at 7:31
0
$\begingroup$

Z-scores are estimates scaled by the estimated standard error of those estimates. Say we have two estimates from two samples $$Z_1 = \frac{\bar{X}_1}{\sigma\sqrt{n_1}} = \frac{1}{\sigma\sqrt{n_1}} \sum_{k=1}^{n_1} X_{1k} \\ Z_2 = \frac{\bar{X}_2}{\sigma\sqrt{n_2}} = \frac{1}{\sigma\sqrt{n_2}} \sum_{k=1}^{n_2} X_{2k} $$

where $\sigma$ is the error in the individual observations $X_{ik}$ (which for simplicity we assume as the same in both groups) and $n_1$ and $n_2$ are the sample sizes.

Here you see that if you would simply add $Z_1$ and $Z_2$ together with equal weights then the individual observations $X_{ik}$ do not get equal weights which is less efficient.

$$\sqrt{0.5} Z_1 + \sqrt{0.5} Z_1 = \sum_{i \in [1,2]} \sum_{k=1}^{n_i} \left( \frac{\sqrt{0.5}}{\sigma\sqrt{n_i}} X_{ik} \right)$$


The justification is that a weighted mean will have a smaller standard error, in comparison to an unweighted arithmetic mean. (From a different perspective the justification is a larger mean z-score instead of smaller standard error, see at the end)

This relates a bit to generalized least squares, and weighted least squares, which have the task to compute an estimate when the observed data points do not have equal variance/error.

What you are doing is computing an average which is a linear estimator and generalized least squares, which will use a weighted mean based on the variance of the individual terms, is the best linear unbiased estimator.


Example of a weighted mean having lower variance:

Example: If you have two observations distributed as

$$\bar{x}_1 \sim N(\mu, \sigma_1^2)\\ \bar{x}_2 \sim N(\mu, \sigma_2^2)$$

then the weighted mean (with weights $a_1+a_2=1$) will be distributed as

$$a_1 z_1 +a_2 z_2 \sim N(\mu, \sigma^2)$$

with the variance a weighted sum $$\sigma^2 = a_1^2 \sigma_1^2 + a_2^2 \sigma_2^2 = \sigma_1^2 -2 a_2 \sigma_1^2 + a_2^2 (\sigma_2^2+\sigma_1^2)$$

which has a minimum in $a_2 = \frac{\sigma_1^2}{\sigma_1^2+\sigma_2^2}$.


Intuitively:

Imagine you have a study with sample size 1000 and a study with sample size 10. The estimate from the latter study has a very large error. When you would average the results of both fifty-fifty then the propagation of error from the inaccurate study will count fifty percent and will lead to a large error in the end result.

The unweighted average of two numbers, one with small error and one with large error, will not have a small error but a medium error. So the average makes the situation worse (because you already had a number with a small error).


Is this because the variance of the z-score is proportional to 1/n, where n is the sample size, so the inverse variance is proportional to n?

The z-scores have variance 1, because they are normalised, but they have different means.

The z-score is roughly distributed as $N(\mu/\sqrt{n},1)$. Where $\mu$ is the population mean. If $\mu$ is non-zero then larger samples will have larger z-scores and that is why you want to give them a stronger weight.

So the comparison with GLS above, which is about different variance instead of different means is a bit twisted, but superficially the principle is related. If you compute the z-scores back to means of the population, then the GLS comparison counts and the goal is to get a linear sum that estimates the population mean and has the lowest variance possible (smaller variance means a larger z-score).

$\endgroup$
3
  • $\begingroup$ This has nothing to do with least squares $\endgroup$
    – Spätzle
    Mar 28 at 6:59
  • $\begingroup$ @Spätzle this has to do with computing an average which is a linear estimator (and unbiased if the individual z-scores are unbiased) and GLS is the best linear unbiased estimator. $\endgroup$ Mar 28 at 7:21
  • $\begingroup$ @Spätzle, OP's question is about en.wikipedia.org/wiki/Inverse-variance_weighting ... " to minimize the variance of the weighted average." $\endgroup$
    – seanv507
    Mar 28 at 8:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.