# UMVU estimator for non-linear transformation of a parameter

Let $$X_1, ..., X_n$$ be iid. and $$X_1\sim N(\mu,1)$$. $$\gamma(\mu)=e^{t\mu}$$ for $$t\neq 0$$

My question is how to find an UMVU estimator for $$\gamma(\mu)$$

My concern is not so much about the specific problem but rather about how to approach this kind of estimation problem in general, if one wants to estimate a non-linear transformation of a parameter. The problem for me is that $$\overline{X}_n$$ is UMVUE for $$\mu$$ but I think that: $$E_\theta[e^{\overline{X}_n}]\neq e^{t\theta}$$.

We never covered this kind of problem in our lecture so I dont really have an idea of how to solve this kind of estimation problem

EDIT: I think I solved it myself: Let $$\frac{1}{k}:=\frac{1}{(2\pi)^{n/2}}$$

$$\int e^{t\overline{X}_n} \cdot \frac{1}{k}\exp(-\frac{1}{2}\sum_{i=1}^n(x_i-\theta)^2)d(x_1, ..., x_n)=$$

$$=\int \frac{1}{k}\exp(-\frac{1}{2}\sum_{i=1}^n(x_i-2(\theta+\frac{t}{n})x_i+\theta^2)d(x_1, ..., x_n)=$$

$$=\exp(t\theta+\frac{t^2}{2n})\int \frac{1}{k}\exp(-\frac{1}{2}\sum_{i=1}^n(x_i-2(\theta+\frac{t}{n})x_i+(\theta+\frac{t}{n})^2)d(x_1, ..., x_n)=$$

$$=\exp(t\theta+\frac{t^2}{2n})$$

The UMVU estimator is therefore $$T(X1, ..., X_n)=\exp(-t^2/2n) \exp(t\overline{X}_n)$$ It is UMVU because $$\overline{X}_n$$ is sufficient and thereby by Lehmann-Scheffe the estimator is UMVU, as $$T(X_1, ... X_n)=g(\overline{X}_n)$$

Remark: I would still have to proof that $$Var_\theta(g(\overline{X}_n)) < \infty \forall \theta$$ for Lehmann-Scheffe to be applicable.

• Non linear transformation of a an unbiased estimate is usually biased, therefore it is natural that a transformation of sample mean doesn't work. Non-linear transformations of efficient estimations also end up being less efficient as far as I know. Not a helpful comment but some boundaries to know about. – Cagdas Ozgenc Jul 13 '18 at 12:52
• Regarding your final conclusion: $\bar X_n$ is a function of the complete sufficient statistic $T=\sum X_i$. That is why if any function of $T$ is unbiased for $\gamma (\mu)$, it is the UMVUE of $\gamma (\mu)$. This is courtesy of the Lehmann -Scheffe theorem. – StubbornAtom Jul 14 '18 at 14:27
• Re your 'remark': The fact that the variance is finite is implied. You can find the variance using the MGF of $\bar X_n$ and see that it is finite, but I don't think one has to do that for the question at hand. – StubbornAtom Jul 17 '18 at 11:43
• I am unfortunately not familiar with MGF so i was not aware of that. – Sebastian Jul 17 '18 at 12:32
• @0rangetree You just have to evaluate couple of integrals to find the moments. – StubbornAtom Jul 17 '18 at 16:30

Your final answer is not quite right. The conclusion due to the sample mean $\bar X$ being only sufficient for $\mu$ also looks faulty.

Recall that $T(X_1,X_2,\cdots,X_n)=\sum_{i=1}^n X_i$ is a complete sufficient statistic for $\mu$.

It is easy to see this if you work with the exponential family setup.

Now we know the distribution of $\bar X$, namely $\bar X\sim\mathcal N\left(\mu,\frac{1}{n}\right)$.

From the moment generating function of a univariate normal distribution, it follows that

$$E_{\mu}(e^{t\bar X})=\exp\left(\mu t+\frac{t^2}{2n}\right)$$

That is,

$$E_{\mu}(e^{t\bar X-t^2/2n})=e^{\mu t}$$

So an unbiased estimator of $e^{\mu t}$ is \begin{align}h(T)&=\exp\left(t\bar X-t^2/2n\right) \\&=\exp\left(T\frac{t}{n}-\frac{t^2}{2n}\right) \end{align}

$h(T)$ is a function of the complete sufficient statistic $T$.

Hence by the Lehmann-Scheffe theorem, $h(T)$ is the UMVUE of $e^{\mu t}$.

You ask how to approach problems regarding estimation of a non-linear transformation of the parameter of interest. I think this is pretty much the same as estimating any function of the parameter.

The usual tools to find a UMVUE (if it exists) are the Rao-Blackwell theorem and/or the Lehmann-Scheffe theorem. One needs to find a complete sufficient statistic ($T$, say), if it exists, for some $\theta$ (which may well be a vector) that parametrises the given population distribution. The first step is to find an unbiased estimator (if it exists) of $\theta$. Now if this unbiased estimator is a function of the complete sufficient statistic, then it will be the UMVUE of $\theta$. This is a corollary of the Lehmann-Scheffe theorem. Even if the unbiased estimator is not a function of the complete sufficient statistic, we have the Rao-Blackwell theorem at hand. This says that if we take any trivial unbiased estimator ($h$, say) of $\theta$, then the conditional expectation $E(h\mid T)$ is the UMVUE of $\theta$. Finding this conditional expectation explicitly might not be an easy task in general.

The remark by @CagdasOzgenc in the comments is worth noticing.

If $U$ is unbiased for $\theta$, then we do not expect $g(U)$ to be unbiased for $g(\theta)$ for any $g$. That approach would just not work in general. One way to see this is Jensen's inequality: $E(g(U))\ge g(E(U))$ for convex $g$ (inequality reverses when $g$ is concave). Note that equality holds in Jensen's inequality if $g$ is a constant function or if $g$ is an affine function.

• ah thanks I had 2 typos but except for that we obtain the same estimator. – Sebastian Jul 14 '18 at 15:42
• but i dont get what you mean by "The conclusion due to the sample mean $\overline{X}$ being only sufficient for $\mu$ also looks fault". I didnt mention Lehmann-Scheffe because i thought it was obvious. Is this what you mean? – Sebastian Jul 14 '18 at 15:47
• @0rangetree I meant your final conclusion (before the edit) that $\bar X$ is UMVUE because it is sufficient is not entirely correct. – StubbornAtom Jul 14 '18 at 15:51

A more general perspective on the question is that most non-linear transforms of parameters $$\theta$$ associated with an unbiased estimator cannot be unbiasedly estimated.

There are indeed many instances in the literature about the impossibility to find an unbiased estimator:

• "A Class of Parameter Functions for Which the Unbiased Estimator Does Not Exist" by Shande Chen, IMS Lecture Notes-Monograph Series Vol. 43, Crossing Boundaries: Statistical Essays in Honor of Jack Hall (2003), pp. 159-164
• "On the Non-Existence of Unbiased Estimators in Constrained Estimation Problems" by Anelia Somekh-Baruch, Amir Leshem, Venkatesh Saligrama (2016), arXiv:1609.07415
• "On nonnegative unbiased estimators", by Pierre E. Jacob, Alexandre H. Thiery (2013)11, arXiv:1309.6473
• Lehmann. E.L. and Casella, G. (Theory of Point Estimation, 1999) have a section [2.4] on this issue, including the case of the standard deviation $$\sigma$$ for which no unbiased estimator based on a fixed sample size exists.

And an extract from C.R. Rao's piece on Erich Lehmann's contribution [9] to the topic, in Selected Works of E.L. Lehmann:

On the side, note that sufficiency and completeness are notions that remain invariant under bijective changes of parameters.