1
$\begingroup$

Consider two random unbiased estimates $\hat X_1,$, $\hat X_2$ of a parameter (complex number) $x$, with estimation errors $E_1 = \hat X_1-x$, $E_2 = \hat X_2-x$.

If the random variables $E_1$, $E_2$ are i.i.d., it is well known that averaging the two estimates, $\hat X = (X_1+X_2)/2$, achieves a smaller error variance (by a factor 1/2) than that of $\hat X_1$ or $\hat X_2$.

If $E_1$, $E_2$ are independent with different variances, the linear combination $\hat X = w_1 X_1 + w_2 X_2$ that minimizes the error is no longer given by $w_1, w_2=1/2$, but it is easy to compute the optimal weights in terms of the error variances. This is also well known, I think.

Lastly, if $E_1$, $E_2$ are correlated with covariance matrix $\mathbf C$, I have obtained the weights that minimize the error variance, subject to the condition that $\hat X$ retains unbiased. Specifically (unless I have made some mistake), $$ w_1 = \frac{C_{22} - \mathrm{Re}\, C_{12}}{C_{11}+C_{22}-2\,\mathrm{Re}\, C_{12}}, \\ w_2 = \frac{C_{11} - \mathrm{Re}\, C_{12}}{C_{11}+C_{22}-2\,\mathrm{Re}\, C_{12}}, $$ where "$\mathrm{Re}$" denotes "real part"; and the variance of the resulting error is indeed smaller than that in each individual estimate.

My question is: is this result for the correlated case well known? Is there some theorem, perhaps generalizing this to more than $2$ estimates? Or any name or pointer I could use to search for this?

I want to use this result, and I would rather reference it if possible than reinvent the wheel.

Improve this question
$\endgroup$
3
  • $\begingroup$ As you, say one can easily compute optimal weights for independent estimators and can also be done for correlated estimators, but that's only if you know their properties (bias, variance). Not sure whether that's a well-known results. However, in practice, I've not seen such results used (e.g. on Kaggle), I assume in part because we need to estimate properties via cross-validation, more complex metrics are often used and solving the problem numerically is relatively straightforward (and you might want regularization towards a simple average). $\endgroup$
    – Björn
    Jul 13, 2022 at 14:24
  • $\begingroup$ Yes, this is well known. There's no need to invoke complex numbers--just separate the variables into their real and imaginary parts at the outset. The weights can be easily derived from the results you cite by first finding linear combinations of the components of the $E_i$ that are uncorrelated (also a well known basic procedure). This result for arbitrarily many random variables is a key ingredient in the Capital Asset Pricing Model (CAPM), which (among other things) determines what combination of correlated financial instruments will have the least volatility (variance). $\endgroup$
    – whuber ♦
    Jul 13, 2022 at 14:37
  • $\begingroup$ @whuber Thank you, I'll use that name in my search $\endgroup$
    – Luis Mendo
    Jul 13, 2022 at 14:46

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.