Which is a better estimator, averaged functions vs. A function of an average? Problem:
Assume that we want to estimate $f(\theta)$ with a pre-specified strictly increasing function $f$ and a parameter $\theta$.
Let $\hat{\theta}_1$ and $\hat{\theta}_2$ be unbiased estimators for $\theta$. My question is to compare which estimator is better for $f(\theta)$:


*

*$f\left(\frac{\hat{\theta}_1 + \hat{\theta}_2}{2}\right)$, or

*$\frac{f(\hat{\theta}_1) + f(\hat{\theta}_2)}{2}$.


My approach:
My approach is taking an expectation on both based on Taylor's expansion of $f$, provided that they are well-defined.


*

*$$\mathbb{E}f\left(\frac{\hat{\theta}_1 + \hat{\theta}_2}{2}\right) = 
f(\theta) + \frac{f''(\theta)}{2}\text{Var}\left(\frac{\hat{\theta}_1 + \hat{\theta}_2}{2}\right) + \ldots
$$

*$$\mathbb{E} \left[\frac{f(\hat{\theta}_1) + f(\hat{\theta}_2)}{2} \right] = 
f(\theta) + \frac{f''(\theta)}{2}\left(\frac{\text{Var}(\hat{\theta}_1) + \text{Var}(\hat{\theta}_2)}{2}\right) + \ldots
$$
Here, the first-order disappears in each expectation because $\hat{\theta}_1$ and $\hat{\theta}_2$ are unbiased.
Let's further assume $f''(\theta)>0$. Then, since
$$\label{varineq}
(0 \le)
\text{Var}\left(\frac{\hat{\theta}_1 + \hat{\theta}_2}{2}\right) \le
\frac{\text{Var}(\hat{\theta}_1) + \text{Var}(\hat{\theta}_2)}{2},
$$
the first estimator (i.e. a function of averages) seems less distant from $f(\theta)$, as long as the remaining terms (order $\ge 3$) are ignored.
The last inequality comes from a simple observation:
$$
\frac{\text{Var}(\hat{\theta}_1) + \text{Var}(\hat{\theta}_2)}{2} - 
\text{Var}\left(\frac{\hat{\theta}_1 + \hat{\theta}_2}{2}\right) =
\dfrac{1}{4}\left( \text{Var}(\hat{\theta}_1) + \text{Var}(\hat{\theta}_2) - 
2\text{Cov}\left(\hat{\theta}_1, \hat{\theta}_2\right) \right) = \dfrac{\text{Var}(\hat{\theta}_1 - \hat{\theta}_2)}{4} \ge 0.
$$
Question:
I wonder if this is logical and there is another way to justify a better estimator.
Edit:
Let's restrict the class of $f$ by the strictly increasing ones.
 A: Suppose $\theta_1$ and $\theta_2$ are unbiased estimators of $\mu$ with a bivariate normal distribution.
Let $\sigma_1 = k \sigma_2$, with $k<1$, so $\theta_1$ is the more efficient estimator.
Let the correlation be $r$, with $-1<r<1$.
We seek the least-variance linear estimates for $\mu$ and $\mu^2$. The results are:
\begin{align}
\min_{a,b}Var[\mu-(a\theta_1+b\theta_2)] \text{ has }&\mu \sim \frac{(1-kr)\theta_1+(k^2-kr)\theta_2}{1-2kr+k^2}\\
\min_{c,d,e}Var[\mu^2-(c\theta_1^2+d\theta_1\theta_2+e\theta_2^2)] \text { has }&\mu^2 \sim \left(\frac{(1-kr)\theta_1+(k^2-kr)\theta_2}{1-2kr+k^2}\right)^{\!2}
\end{align}
So for functions $f$ that are quadratic or well approximated by quadratics, if $\frac{\theta_1+\theta_2}{2}$ is the best linear estimate for $\mu$, then $f\left(\frac{\theta_1+\theta_2}{2}\right)$ is the best linear estimate for $f(\mu)$. In general an appropriate weighted average inside $f$ works even better.
Example: $\theta_1$ is the mean of a sample from a normal parent with mean $\mu$, and $\theta_2$ is the median from the same sample. Then $k=r=\sqrt{2/\pi}$, and the above formula shows that the best estimate for $\mu$ is $\theta_1$, and the best estimate for $\mu^2$ is $\theta_1^2$, i.e. ignoring the median.
Example: $\theta_1$ is the mean of a sample from a normal parent with mean $\mu$, and $\theta_2$ is the median of a different sample of the same size. Then $k=\sqrt{2/\pi}$, $r=0$, so the best estimate for $\mu$ is $(\pi\theta_1+2\theta_2)/(\pi+2)$, or $.611\theta_1 + .389\theta_2$, and the best estimate for $\mu^2$ is the square of that.
Example: $\theta_1$ is the mean of a sample from a normal parent with mean $\mu$, and $\theta_2$ is the mean of a different sample of the same size. Then $k=1$, $r=0$, so the best estimate for $\mu$ is $(\theta_1+\theta_2)/2$, and the best estimate for $\mu^2$ is $\left(\frac{\theta_1+\theta_2}2\right)^{\!2}$.
