9
$\begingroup$

Let $X_1, X_2, . . . , X_n$ be iid random variables having pdf

$$f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$$

where $\theta >0$. Give the UMVUE of $\frac{1}{\theta}$ and compute its variance

I have learned about two such methods for obtained UMVUE's:

  • Cramer-Rao Lower Bound (CRLB)
  • Lehmann-Scheffe Thereom

I am going to attempt this using the former of the two. I must admit that I do not completely understand what is going on here, and I'm basing my attempted solution off of an example problem. I have that $f_X(x\mid\theta)$ is a full one-parameter exponential family with

$h(x)=I_{(0,\infty)}$, $c(\theta)=\theta$, $w(\theta)=-(1+\theta)$, $t(x)=\text{log}(1+x)$

Since $w'(\theta)=1$ is nonzero on $\Theta$, the CRLB result applies. We have

$$\text{log }f_X(x\mid\theta)=\text{log}(\theta)-(1+\theta)\cdot\text{log}(1+x)$$

$$\frac{\partial}{\partial \theta}\text{log }f_X(x\mid\theta)=\frac{1}{\theta}-\text{log}(1+x)$$

$$\frac{\partial^2}{\partial \theta^2}\text{log }f_X(x\mid\theta)=-\frac{1}{\theta^2}$$

so $$I_1(\theta)=-\mathsf E\left(-\frac{1}{\theta^2}\right)=\frac{1}{\theta^2}$$

and the CRLB for unbiased estimators of $\tau(\theta)$ is

$$\frac{[\tau'(\theta)]^2}{n\cdot I _1(\theta)} = \frac{\theta^2}{n}[\tau'(\theta)]^2$$

Since $$\sum_{i=1}^n t(X_i)=\sum_{i=1}^n \text{log}(1+X_i)$$

then any linear function of $\sum_{i=1}^n \text{log}(1+X_i)$, or equivalently, any linear function of $\frac{1}{n}\sum_{i=1}^n \text{log}(1+X_i)$, will attain the CRLB of its expectation, and thus be a UMVUE of its expectation. Since $\mathsf E(\text{log}(1+X))=\frac{1}{\theta}$ we have that the UMVUE of $\frac{1}{\theta}$ is $\frac{1}{n}\sum_{i=1}^n \text{log}(1+X_i)$

For a natural parameterization we can let $\eta=-(1+\theta)\Rightarrow \theta=-(\eta+1)$

Then

$$\mathsf{Var}(\text{log}(1+X))=\frac{d}{d\eta}\left(-\frac{1}{\eta+1}\right)=\frac{1}{(\eta+1)^2}=\frac{1}{\theta^2}$$

Is this a valid solution? Is there a more simple approach? Does this method only work when the $\mathsf E(t(x))$ equals what you're trying to estimate?

$\endgroup$
  • 4
    $\begingroup$ At the point where you showed that the pdf is a member of the one-parameter exponential family, it is immediately clear that a complete sufficient statistic for the family is $$T(X_1,\ldots,X_n)=\sum_{i=1}^n\ln(1+X_i)$$ Since, as you say, $E(T/n)=\frac{1}{\theta}$, $T/n$ is the UMVUE of $1/\theta$ by the Lehmann-Scheffe theorem. $\endgroup$ – StubbornAtom Oct 24 '18 at 18:41
  • $\begingroup$ So the part where I have "Since $w'(\theta)=1$ is nonzero.....$\frac{\theta^2}{n}[\tau'(\theta)]^2$" is irrelevant? $\endgroup$ – Remy Oct 24 '18 at 18:46
  • 2
    $\begingroup$ Not really; the variance of $T$ is easier to find using the CRLB. So to solve both questions at once, your argument is sufficient. $\endgroup$ – StubbornAtom Oct 24 '18 at 18:48
  • $\begingroup$ To find the variance that way, would I take $\frac{\theta^2}{n}[\tau'(\theta)]^2=\frac{\theta^2}{n}\left(-\frac{1}{\theta^2}\right)^2=\frac{1}{n\theta^2}$? Hence, I did it incorrectly previously? $\endgroup$ – Remy Oct 24 '18 at 18:56
  • $\begingroup$ Yes, that is the variance of $T$. Precisely. $\endgroup$ – StubbornAtom Oct 24 '18 at 18:58
7
$\begingroup$

Your reasoning is mostly correct.

The joint density of the sample $(X_1,X_2,\ldots,X_n)$ is

\begin{align} f_{\theta}(x_1,x_2,\ldots,x_n)&=\frac{\theta^n}{\left(\prod_{i=1}^n (1+x_i)\right)^{1+\theta}}\mathbf1_{x_1,x_2,\ldots,x_n>0}\qquad,\,\theta>0 \\\\\implies \ln f_{\theta}(x_1,x_2,\ldots,x_n)&=n\ln(\theta)-(1+\theta)\sum_{i=1}^n\ln(1+x_i)+\ln(\mathbf1_{\min_{1\le i\le n} x_i>0}) \\\\\implies\frac{\partial}{\partial \theta}\ln f_{\theta}(x_1,x_2,\ldots,x_n)&=\frac{n}{\theta}-\sum_{i=1}^n\ln(1+x_i) \\\\&=-n\left(\frac{\sum_{i=1}^n\ln(1+x_i)}{n}-\frac{1}{\theta}\right) \end{align}

Thus we have expressed the score function in the form

$$\frac{\partial}{\partial \theta}\ln f_{\theta}(x_1,x_2,\ldots,x_n)=k(\theta)\left(T(x_1,x_2,\ldots,x_n)-\frac{1}{\theta}\right)\tag{1}$$

, which is the equality condition in the Cramér-Rao inequality.

It is not difficult to verify that $$E(T)=\frac{1}{n}\sum_{i=1}^n\underbrace{E(\ln(1+X_i))}_{=1/\theta}=\frac{1}{\theta}\tag{2}$$

From $(1)$ and $(2)$ we can conclude that

  • The statistic $T(X_1,X_2,\ldots,X_n)$ is an unbiased estimator of $1/\theta$.
  • $T$ satisfies the equality condition of the Cramér-Rao inequality.

These two facts together imply that $T$ is the UMVUE of $1/\theta$.

The second bullet actually tells us that variance of $T$ attains the Cramér-Rao lower bound for $1/\theta$.

Indeed, as you have shown, $$E_{\theta}\left[\frac{\partial^2}{\partial\theta^2}\ln f_{\theta}(X_1)\right]=-\frac{1}{\theta^2}$$

This implies that the information function for the whole sample is $$I(\theta)=-nE_{\theta}\left[\frac{\partial^2}{\partial\theta^2}\ln f_{\theta}(X_1)\right]=\frac{n}{\theta^2}$$

So the Cramér-Rao lower bound for $1/\theta$ and hence the variance of the UMVUE is

$$\operatorname{Var}(T)=\frac{\left[\frac{d}{d\theta}\left(\frac{1}{\theta}\right)\right]^2}{I(\theta)}=\frac{1}{n\theta^2}$$


Here we have exploited a corollary of the Cramér-Rao inequality, which says that for a family of distributions $f$ parametrised by $\theta$ (assuming regularity conditions of CR inequality to hold), if a statistic $T$ is unbiased for $g(\theta)$ for some function $g$ and if it satisfies the condition of equality in the CR inequality, namely $$\frac{\partial}{\partial\theta}\ln f_{\theta}(x)=k(\theta)\left(T(x)-g(\theta)\right)$$, then $T$ must be the UMVUE of $g(\theta)$. So this argument does not work in every problem.

Alternatively, using the Lehmann-Scheffe theorem you could say that $T=\frac{1}{n}\sum_{i=1}^{n} \ln(1+X_i)$ is the UMVUE of $1/\theta$ as it is unbiased for $1/\theta$ and is a complete sufficient statistic for the family of distributions. That $T$ is compete sufficient is clear from the structure of the joint density of the sample in terms of a one-parameter exponential family. But variance of $T$ might be a little tricky to find directly.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.