1
$\begingroup$

I'm working on a problem involving two linear unbiased estimators $T$ and $T'$ of a parameter $\theta$, defined from a sample $\{X_1, \dots, X_n\}$ with mean $\theta$ and finite variance. I aim to prove that $\text{Cov}_\theta(T, T') = \text{Var}_\theta(T)$, given that:

  • $T = \sum_{i=1}^{n} \alpha_i X_i$ and $T'$ is another linear unbiased estimator of $\theta$, with $T' = \sum_{i=1}^{n} \beta_i X_i$.
  • Both $T$ and $T'$ are unbiased ($\mathbb{E}[T] = \mathbb{E}[T'] = \theta$).
  • $T$ has minimum variance among all such estimators.

Through my derivations, I reached that $\text{Cov}_\theta(T, T') = \sigma^2 \sum_{i=1}^{n} \alpha_i \beta_i$ and $\text{Var}_\theta(T) = \sigma^2 \sum_{i=1}^{n} \alpha_i^2$. From these, I inferred that if $\text{Cov}_\theta(T, T') = \text{Var}_\theta(T)$, it should imply $\sum_{i=1}^{n} \alpha_i \beta_i = \sum_{i=1}^{n} \alpha_i^2$, but it seems hard to prove, and I am honestly not even sure if this is a solvable equation. I would need to prove this in order to conclude my proof.

I'm unsure if what I got to is universally valid or if I've overlooked critical assumptions about the relationship between $T$ and $T'$. Here's a summary of my steps:

  1. Established the unbiasedness of $T$ and $T'$.
  2. Calculated $\text{Cov}_\theta(T, T')$ using the expectation of their product minus the product of their expectations.
  3. Arrived at the conclusion based on the expressions for covariance and variance.

Is there a flaw in my reasoning, or are there specific conditions under which this relationship holds true? I'm particularly interested in understanding the assumptions required for $\text{Cov}_\theta(T, T') = \text{Var}_\theta(T)$ to be valid. Any insights or references to similar proofs would be greatly appreciated!

$\endgroup$

2 Answers 2

0
$\begingroup$

Consider the variance of $T_a=aT'+(1-a)T$, which (for any $a$) is another linear unbiased estimator

$$\mathrm{var}[T_a]=a^2\mathrm{var}[T']+(1-a)^2\mathrm{var}[T]+2a(1-a)\mathrm{cov}[T,T']$$ or rearranged $$\mathrm{var}[T_a]=a^2\left(\mathrm{var}[T']+\mathrm{var}[T]-2\mathrm{cov}[T,T']\right)+2a(\mathrm{cov}[T,T']-\mathrm{var}[T])+\left(\mathrm{var}[T]\right)$$

By assumption this must be minimised at $a=0$, because $T$ is the best linear unbiased estimator. It's a standard high-school fact about quadratics that the minimum is at $$a_{\text{opt}}=\frac{-2(\mathrm{cov}[T,T']-\mathrm{var}[T])}{2\left(\mathrm{var}[T']+\mathrm{var}[T]-2\mathrm{cov}[T,T']\right)}$$ and for this to be 0, we need $$\mathrm{cov}[T,T']=\mathrm{var}[T]$$

$\endgroup$
1
  • $\begingroup$ Thank you so much, that's quite straightforward and quite a clever way to solve this! $\endgroup$ Feb 15 at 1:58
0
$\begingroup$

Unbiasedness of $T$ and $T'$ imply $\sum_{i=1}^n \alpha_i = \sum_{i=1}^n \beta_i = 1$.

$T$ having minimium variance means $\alpha_1, \ldots, \alpha_n$ minimize $\sum_{i=1}^n \alpha_i^2$ subject to $\sum_{i=1}^n \alpha_i = 1$. By Cauchy-Schwarz, we have $$\left(\sum_{i=1}^n \alpha_i^2 \right)\left(\sum_{i=1}^n 1\right) \ge \left(\sum_{i=1}^n (\alpha_i \cdot 1)\right)^2 = 1,$$ for any $\alpha_1, \ldots, \alpha_n$. For $\sum_{i=1}^n \alpha_i^2$ to be minimized, we need $\alpha_i = 1/n$.

Combining this with the fact that $\sum_{i=1}^n \beta_i = 1$ should allow you to prove $\sum_{i=1}^n \alpha_i^2 = \sum_{i=1}^n \alpha_i \beta_i$.

$\endgroup$
3
  • $\begingroup$ Two quite different proofs with the same basic idea.Nice. $\endgroup$ Feb 14 at 4:23
  • $\begingroup$ Clever, is $\alpha_i = \frac{1}{n}$the unique solution for $T$ to be minimized? $\endgroup$ Feb 15 at 1:59
  • $\begingroup$ Yes. For Cauchy-Schwarz to attain equality in this case, there exists some $c$ such that $\alpha_i = c \cdot 1$ for all $i$. The constraint on $\sum_{i=1}^n \alpha_i$ forces $c=1/n$. @TahaRhaouti $\endgroup$
    – angryavian
    Feb 15 at 2:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.