Show that a linear combination of UMVU estimators is also a UMVU estimator

Question

Suppose I have two estimators $\delta_1$ and $\delta_2$ with finite second moments, and they are UMVU estimators of $f_1(\theta)$ and $f_2(\theta)$, respectively.

Now, for some real numbers $n_1$ and $n_2$, I'm working with the linear combination $n_1\delta_1+n_2\delta_2$. I need to show that this estimator:

1. Also has a finite second moment.
2. Is the UMVU estimator for $n_1f_1(\theta)+n_2f_2(\theta)$

Here's what I have for part 1, checking the second moment:

\begin{align} E((n_1\delta_1+n_2\delta_2)^2) &= E(n_1^2\delta_1^2+2n_1n_2\delta_1\delta_2+n_2^2\delta_2^2) \\ &= n_1^2E(\delta_1^2)+2n_1n_2E(\delta_1\delta_2)+n_2^2E(\delta_2^2) \end{align}

I know that the first and third term above are finite, by the supposition. I'm not sure how to show that the middle term is finite as well. Is my thinking correct?

For finding that this is minimum variance, I'm doing the following:

\begin{align} Var(n_1\delta_1+n_1\delta_2) &= n_1^2Var(\delta_1)+n_2^2Var(\delta_2)+2n_1n_2Cov(\delta_1,\delta_2) \end{align}

Now, since $\delta_1$ and $\delta_2$ are UMVUE, the first two terms above are essentailly minimized. By the covariance inequality,

$$Cov(\delta_1, \delta_2)\le \sqrt{Var(\delta_1)Var(\delta_2)}$$

Again, since $\delta_1$ and $\delta_2$ are UMVUE, the right hand side of the inequality is minimized, and so is the covariance. Is my reasoning correct?

Answer 1

That $n_1\delta_1+n_2\delta_2$ has finite second moment has been shown in the other answer.

To prove the linear combination is UMVUE, I would use this necessary-sufficient condition which states that an unbiased estimator (with finite second moment) is UMVUE if and only if it is uncorrelated with every unbiased estimator of zero.

Let $\mathcal U_0$ be the class of all unbiased estimators of zero with finite variances.

Since $\delta_1$ is UMVUE of its expectation,

$$\operatorname{Cov}_{\theta}(\delta_1,h)=\operatorname E_{\theta}(\delta_1h)=0\quad\forall\,\theta\in\Omega,\,h\in \mathcal U_0$$

Similarly,

$$\operatorname{Cov}_{\theta}(\delta_2,h)=\operatorname E_{\theta}(\delta_2h)=0\quad\forall\,\theta\in\Omega,\,h\in \mathcal U_0$$

Therefore,

$$\operatorname E_{\theta}\left((n_1\delta_1+n_2\delta_2)h\right)=n_1\operatorname E_{\theta}(\delta_1h)+n_2\operatorname E_{\theta}(\delta_2h)=0\quad\forall\,\theta\in\Omega,\,h\in \mathcal U_0$$

That is,

$$\operatorname{Cov}_{\theta}(n_1\delta_1+n_2\delta_2,h)=0\quad\forall\,\theta\in\Omega,\,h\in \mathcal U_0$$

So $n_1\delta_1+n_2\delta_2$ is uncorrelated with every unbiased estimator of zero, proving that it is UMVUE of its expectation $n_1f_1(\theta)+n_2f_2(\theta)$.

Answer 2

For now this will be a very incomplete answer, addressing only one of the things you asked about.

The question about the middle term should be about whether the expected value of the absolute value of the random variable is finite: $$ \operatorname E(|\delta_1\delta_2|) < +\infty \text{ ?} $$ The answer is that it is, because of the Cauchy–Schwarz inequality: $$ \operatorname E(|\delta_1\delta_2|) \le \sqrt{\operatorname E(\delta_2^2)\operatorname E(\delta_2^2)}. $$

You have \begin{align} & \operatorname E(n_1\delta_1 + n_2\delta_2) \\[8pt] = {} & n_1\operatorname E(\delta_1) + n_2\operatorname E(\delta_2) \\[8pt] = {} & n_1 f_1(\theta) + n_2 f_2(\theta), \end{align} so this is unbiased.