UMVU estimator for non-linear transformation of a parameter

Question

Let $X_1, ..., X_n$ be iid. and $X_1\sim N(\mu,1)$. $\gamma(\mu)=e^{t\mu}$ for $t\neq 0$

My question is how to find an UMVU estimator for $\gamma(\mu)$

My concern is not so much about the specific problem but rather about how to approach this kind of estimation problem in general, if one wants to estimate a non-linear transformation of a parameter. The problem for me is that $\overline{X}_n$ is UMVUE for $\mu$ but I think that: $E_\theta[e^{\overline{X}_n}]\neq e^{t\theta}$.

We never covered this kind of problem in our lecture so I dont really have an idea of how to solve this kind of estimation problem

EDIT: I think I solved it myself: Let $\frac{1}{k}:=\frac{1}{(2\pi)^{n/2}}$

$ \int e^{t\overline{X}_n} \cdot \frac{1}{k}\exp(-\frac{1}{2}\sum_{i=1}^n(x_i-\theta)^2)d(x_1, ..., x_n)= $

$=\int \frac{1}{k}\exp(-\frac{1}{2}\sum_{i=1}^n(x_i-2(\theta+\frac{t}{n})x_i+\theta^2)d(x_1, ..., x_n)= $

$=\exp(t\theta+\frac{t^2}{2n})\int \frac{1}{k}\exp(-\frac{1}{2}\sum_{i=1}^n(x_i-2(\theta+\frac{t}{n})x_i+(\theta+\frac{t}{n})^2)d(x_1, ..., x_n)=$

$=\exp(t\theta+\frac{t^2}{2n})$

The UMVU estimator is therefore $T(X1, ..., X_n)=\exp(-t^2/2n) \exp(t\overline{X}_n)$ It is UMVU because $\overline{X}_n $ is sufficient and thereby by Lehmann-Scheffe the estimator is UMVU, as $T(X_1, ... X_n)=g(\overline{X}_n)$

Remark: I would still have to proof that $Var_\theta(g(\overline{X}_n)) < \infty \forall \theta $ for Lehmann-Scheffe to be applicable.

Answer 1

Your final answer is not quite right. The conclusion due to the sample mean $\bar X$ being only sufficient for $\mu$ also looks faulty.

Recall that $T(X_1,X_2,\cdots,X_n)=\sum_{i=1}^n X_i$ is a complete sufficient statistic for $\mu$.

It is easy to see this if you work with the exponential family setup.

Now we know the distribution of $\bar X$, namely $\bar X\sim\mathcal N\left(\mu,\frac{1}{n}\right)$.

From the moment generating function of a univariate normal distribution, it follows that

$$E_{\mu}(e^{t\bar X})=\exp\left(\mu t+\frac{t^2}{2n}\right)$$

That is,

$$E_{\mu}(e^{t\bar X-t^2/2n})=e^{\mu t}$$

So an unbiased estimator of $e^{\mu t}$ is \begin{align}h(T)&=\exp\left(t\bar X-t^2/2n\right) \\&=\exp\left(T\frac{t}{n}-\frac{t^2}{2n}\right) \end{align}

$h(T)$ is a function of the complete sufficient statistic $T$.

Hence by the Lehmann-Scheffe theorem, $h(T)$ is the UMVUE of $e^{\mu t}$.

---

You ask how to approach problems regarding estimation of a non-linear transformation of the parameter of interest. I think this is pretty much the same as estimating any function of the parameter.

The usual tools to find a UMVUE (if it exists) are the Rao-Blackwell theorem and/or the Lehmann-Scheffe theorem. One needs to find a complete sufficient statistic ($T$, say), if it exists, for some $\theta$ (which may well be a vector) that parametrises the given population distribution. The first step is to find an unbiased estimator (if it exists) of $\theta$. Now if this unbiased estimator is a function of the complete sufficient statistic, then it will be the UMVUE of $\theta$. This is a corollary of the Lehmann-Scheffe theorem. Even if the unbiased estimator is not a function of the complete sufficient statistic, we have the Rao-Blackwell theorem at hand. This says that if we take any trivial unbiased estimator ($h$, say) of $\theta$, then the conditional expectation $E(h\mid T)$ is the UMVUE of $\theta$. Finding this conditional expectation explicitly might not be an easy task in general.

The remark by @CagdasOzgenc in the comments is worth noticing.

If $U$ is unbiased for $\theta$, then we do not expect $g(U)$ to be unbiased for $g(\theta)$ for any $g$. That approach would just not work in general. One way to see this is Jensen's inequality: $E(g(U))\ge g(E(U))$ for convex $g$ (inequality reverses when $g$ is concave). Note that equality holds in Jensen's inequality if $g$ is a constant function or if $g$ is an affine function.

Answer 2

A more general perspective on the question is that most non-linear transforms of parameters $\theta$ associated with an unbiased estimator cannot be unbiasedly estimated.

There are indeed many instances in the literature about the impossibility to find an unbiased estimator:

• "A Class of Parameter Functions for Which the Unbiased Estimator Does Not Exist" by Shande Chen, IMS Lecture Notes-Monograph Series Vol. 43, Crossing Boundaries: Statistical Essays in Honor of Jack Hall (2003), pp. 159-164
• "On the Non-Existence of Unbiased Estimators in Constrained Estimation Problems" by Anelia Somekh-Baruch, Amir Leshem, Venkatesh Saligrama (2016), arXiv:1609.07415
• "On nonnegative unbiased estimators", by Pierre E. Jacob, Alexandre H. Thiery (2013)11, arXiv:1309.6473
• Lehmann. E.L. and Casella, G. (Theory of Point Estimation, 1999) have a section [2.4] on this issue, including the case of the standard deviation $\sigma$ for which no unbiased estimator based on a fixed sample size exists.

And an extract from C.R. Rao's piece on Erich Lehmann's contribution [9] to the topic, in Selected Works of E.L. Lehmann:

On the side, note that sufficiency and completeness are notions that remain invariant under bijective changes of parameters.