Let $X_1, ..., X_n$ be iid. and $X_1\sim N(\mu,1)$. $\gamma(\mu)=e^{t\mu}$ for $t\neq 0$

My question is how to find an UMVU estimator for $\gamma(\mu)$

My concern is not so much about the specific problem but rather about how to approach this kind of estimation problem in general, if one wants to estimate a non-linear transformation of a parameter. The problem for me is that $\overline{X}_n$ is UMVUE for $\mu$ but I think that: $E_\theta[e^{\overline{X}_n}]\neq e^{t\theta}$.

We never covered this kind of problem in our lecture so I dont really have an idea of how to solve this kind of estimation problem

EDIT: I think I solved it myself: Let $\frac{1}{k}:=\frac{1}{(2\pi)^{n/2}}$

$ \int e^{t\overline{X}_n} \cdot \frac{1}{k}\exp(-\frac{1}{2}\sum_{i=1}^n(x_i-\theta)^2)d(x_1, ..., x_n)= $

$=\int \frac{1}{k}\exp(-\frac{1}{2}\sum_{i=1}^n(x_i-2(\theta+\frac{t}{n})x_i+\theta^2)d(x_1, ..., x_n)= $

$=\exp(t\theta+\frac{t^2}{2n})\int \frac{1}{k}\exp(-\frac{1}{2}\sum_{i=1}^n(x_i-2(\theta+\frac{t}{n})x_i+(\theta+\frac{t}{n})^2)d(x_1, ..., x_n)=$


The UMVU estimator is therefore $T(X1, ..., X_n)=\exp(-t^2/2n) \exp(t\overline{X}_n)$ It is UMVU because $\overline{X}_n $ is sufficient and thereby by Lehmann-Scheffe the estimator is UMVU, as $T(X_1, ... X_n)=g(\overline{X}_n)$

Remark: I would still have to proof that $Var_\theta(g(\overline{X}_n)) < \infty \forall \theta $ for Lehmann-Scheffe to be applicable.

  • 3
    $\begingroup$ Non linear transformation of a an unbiased estimate is usually biased, therefore it is natural that a transformation of sample mean doesn't work. Non-linear transformations of efficient estimations also end up being less efficient as far as I know. Not a helpful comment but some boundaries to know about. $\endgroup$ – Cagdas Ozgenc Jul 13 '18 at 12:52
  • $\begingroup$ Regarding your final conclusion: $\bar X_n$ is a function of the complete sufficient statistic $T=\sum X_i$. That is why if any function of $T$ is unbiased for $\gamma (\mu)$, it is the UMVUE of $\gamma (\mu)$. This is courtesy of the Lehmann -Scheffe theorem. $\endgroup$ – StubbornAtom Jul 14 '18 at 14:27
  • $\begingroup$ Re your 'remark': The fact that the variance is finite is implied. You can find the variance using the MGF of $\bar X_n$ and see that it is finite, but I don't think one has to do that for the question at hand. $\endgroup$ – StubbornAtom Jul 17 '18 at 11:43
  • $\begingroup$ I am unfortunately not familiar with MGF so i was not aware of that. $\endgroup$ – Sebastian Jul 17 '18 at 12:32
  • $\begingroup$ @0rangetree You just have to evaluate couple of integrals to find the moments. $\endgroup$ – StubbornAtom Jul 17 '18 at 16:30

Your final answer is not quite right. The conclusion due to the sample mean $\bar X$ being only sufficient for $\mu$ also looks faulty.

Recall that $T(X_1,X_2,\cdots,X_n)=\sum_{i=1}^n X_i$ is a complete sufficient statistic for $\mu$.

It is easy to see this if you work with the exponential family setup.

Now we know the distribution of $\bar X$, namely $\bar X\sim\mathcal N\left(\mu,\frac{1}{n}\right)$.

From the moment generating function of a univariate normal distribution, it follows that

$$E_{\mu}(e^{t\bar X})=\exp\left(\mu t+\frac{t^2}{2n}\right)$$

That is,

$$E_{\mu}(e^{t\bar X-t^2/2n})=e^{\mu t}$$

So an unbiased estimator of $e^{\mu t}$ is \begin{align}h(T)&=\exp\left(t\bar X-t^2/2n\right) \\&=\exp\left(T\frac{t}{n}-\frac{t^2}{2n}\right) \end{align}

$h(T)$ is a function of the complete sufficient statistic $T$.

Hence by the Lehmann-Scheffe theorem, $h(T)$ is the UMVUE of $e^{\mu t}$.

You ask how to approach problems regarding estimation of a non-linear transformation of the parameter of interest. I think this is pretty much the same as estimating any function of the parameter.

The usual tools to find a UMVUE (if it exists) are the Rao-Blackwell theorem and/or the Lehmann-Scheffe theorem. One needs to find a complete sufficient statistic ($T$, say), if it exists, for some $\theta$ (which may well be a vector) that parametrises the given population distribution. The first step is to find an unbiased estimator (if it exists) of $\theta$. Now if this unbiased estimator is a function of the complete sufficient statistic, then it will be the UMVUE of $\theta$. This is a corollary of the Lehmann-Scheffe theorem. Even if the unbiased estimator is not a function of the complete sufficient statistic, we have the Rao-Blackwell theorem at hand. This says that if we take any trivial unbiased estimator ($h$, say) of $\theta$, then the conditional expectation $E(h\mid T)$ is the UMVUE of $\theta$. Finding this conditional expectation explicitly might not be an easy task in general.

The remark by @CagdasOzgenc in the comments is worth noticing.

If $U$ is unbiased for $\theta$, then we do not expect $g(U)$ to be unbiased for $g(\theta)$ for any $g$. That approach would just not work in general. One way to see this is Jensen's inequality: $E(g(U))\ge g(E(U))$ for convex $g$ (inequality reverses when $g$ is concave). Note that equality holds in Jensen's inequality if $g$ is a constant function or if $g$ is an affine function.

  • $\begingroup$ ah thanks I had 2 typos but except for that we obtain the same estimator. $\endgroup$ – Sebastian Jul 14 '18 at 15:42
  • $\begingroup$ but i dont get what you mean by "The conclusion due to the sample mean $\overline{X}$ being only sufficient for $\mu$ also looks fault". I didnt mention Lehmann-Scheffe because i thought it was obvious. Is this what you mean? $\endgroup$ – Sebastian Jul 14 '18 at 15:47
  • $\begingroup$ @0rangetree I meant your final conclusion (before the edit) that $\bar X$ is UMVUE because it is sufficient is not entirely correct. $\endgroup$ – StubbornAtom Jul 14 '18 at 15:51
  • $\begingroup$ @StubbornAtoM: What is the difference between the $T$ and $t$ in the last step? I got lost in the notation. Thanks. $\endgroup$ – user1916067 Sep 23 '20 at 13:59
  • $\begingroup$ @user1916067 $t$ is a constant given in the question; it has nothing to do with the statistic $T$. $\endgroup$ – StubbornAtom Sep 23 '20 at 14:29

A more general perspective on the question is that most non-linear transforms of parameters $\theta$ associated with an unbiased estimator cannot be unbiasedly estimated.

There are indeed many instances in the literature about the impossibility to find an unbiased estimator:

  • "A Class of Parameter Functions for Which the Unbiased Estimator Does Not Exist" by Shande Chen, IMS Lecture Notes-Monograph Series Vol. 43, Crossing Boundaries: Statistical Essays in Honor of Jack Hall (2003), pp. 159-164
  • "On the Non-Existence of Unbiased Estimators in Constrained Estimation Problems" by Anelia Somekh-Baruch, Amir Leshem, Venkatesh Saligrama (2016), arXiv:1609.07415
  • "On nonnegative unbiased estimators", by Pierre E. Jacob, Alexandre H. Thiery (2013)11, arXiv:1309.6473
  • Lehmann. E.L. and Casella, G. (Theory of Point Estimation, 1999) have a section [2.4] on this issue, including the case of the standard deviation $\sigma$ for which no unbiased estimator based on a fixed sample size exists.

And an extract from C.R. Rao's piece on Erich Lehmann's contribution [9] to the topic, in Selected Works of E.L. Lehmann:

enter image description here

On the side, note that sufficiency and completeness are notions that remain invariant under bijective changes of parameters.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.