From 'Modern Mathematical Statistics with Applications' (Devore and Beck) pg 377

Let $X_1, X_2 \ldots$ be a random sample from the disbribution $f(x,\theta) = \theta x^{\theta-1}$ for $x\in {]0,1[}$ and $\theta >0$.

The MLE is given by $\hat \theta = \dfrac{-n}{\sum_i\ln X_i}$.

It also shows how $$\sqrt{n}(\hat \theta - \theta) \stackrel{\text{D}}{\rightarrow}N(0,\theta)$$

Using the fact that the MLE is consistent and the CLT.

I wonder if this estimator is also unbiased, I want to show how $E[\hat \theta] = \theta $. Any ideas (no full solutions please) on how to (dis)prove this.

Here are some things I've tried:

  • Notice how $E[\ln X] = \frac{-1}{\theta}$ or $E[\sum_i \ln X_i]= \dfrac{-n}{\theta}$

  • Calculating the expected value seems cumbersome:

$$E\left[\frac{-n}{\sum_i \ln X_i}\right] = \int_0^1\ldots \int_0^1 \frac{-n}{\sum_i \ln x_i} \cdot \theta^n x_1^{\theta-1}x_2^{\theta-1}\ldots x_n^{\theta-1}\operatorname d x_1\ldots \operatorname dx_n$$

Edit after the responses from JohnK and Alecos Papadopoulos

  • Direct calculation: Okay, this is pretty cool. I worked it out and found how $\sum_i -\ln X_i = \sum Y_i \stackrel{\text{d}}{=} \Gamma(n,\theta)$ (through the hints supplied byJohnK), then I immedialty used LOTUS and found $E[\hat \theta] = \dfrac{n}{n-1}\theta > \theta$.

  • Jensen's reasoning: I guess I should use Jensen's inequality as follows, since $E\left[\sum_i \ln X_i\right] = \dfrac{-n}{\theta}$ and because $\sum_i \ln X_i \in {]-\infty, 0[}$ I should look at the left part of the function $g:x\mapsto \dfrac{-n}{x}$ which is stictly convex. Jensen's concludes: $$E[g(X)] > g(E[X])$$ this would imply here: $$E[\hat \theta] > \theta$$

  • $\begingroup$ What is wrong with the first remark? - woops, noticed the typo :) $\endgroup$ – dietervdf Apr 10 '17 at 14:48
  • $\begingroup$ @AlecosPapadopoulos: Okay, using Jensen's results in $E[\hat \theta] \leqslant \theta$ (since $g: x\mapsto \frac{-n}{x}$ is concave in $]0,1[$), but this doesn't imply unbiasedness? $\endgroup$ – dietervdf Apr 10 '17 at 14:55
  • $\begingroup$ Strict inequality $\endgroup$ – Alecos Papadopoulos Apr 10 '17 at 14:55
  • $\begingroup$ Isn't Jensen's inequality always a weak inequality? $\endgroup$ – dietervdf Apr 10 '17 at 15:07
  • $\begingroup$ No. The function is strictly concave. $\endgroup$ – Alecos Papadopoulos Apr 10 '17 at 15:08

Here is a sequence of steps that will help:

-Find the distribution of $Y=-\log X$

-Find the distribution of $\sum_{i=1}^n Y_i = - \sum_{i=1}^n \log X_i$

-Lastly, evaluate the expectation of $Z = \frac{1}{\bar{Y}}$. You can do this directly using the distribution of the previous step and LOTUS or by first finding the distribution of $Z$.

Hint: the gamma family is a truly large family of distributions.

  • $\begingroup$ - Since $X = e^{-Y}$ one finds $F_Y(y \theta) = e^{-\theta y}$ or $f_Y(y, \theta) = -\theta e^{-\theta y}$ where $y\in {]0, +\infty[}$, can you expand on the distribution of that sum $\sum_i Y_i$? $\endgroup$ – dietervdf Apr 10 '17 at 15:10
  • $\begingroup$ @dietervdf Check your derivation again, that minus sign shouldn't be there. Then for the distribution of the sum, you need to use the additivity property of the gamma family of distributions. If I say more, I will give it away. $\endgroup$ – JohnK Apr 10 '17 at 15:16
  • $\begingroup$ I think I see where you are going, thanks for the hint! Just one thing, why shouldn't the minus be there? My derivation is as follows: $F_X(x,\theta) = x^\theta$ (with $x\in {]0,1[}$, then using the mapping $X = e^{-Y}$ one gets the cdf of $Y$, as $F_Y(y,\theta) = e^{-\theta y}$ which implies $f_Y(y,\theta) = -\theta e^{-\theta y}$ $\endgroup$ – dietervdf Apr 10 '17 at 15:26
  • 1
    $\begingroup$ @dietervdf You you need to take the complement of the CDF as you have a minus there. $\Pr \left( - \log X \leq x \right) = 1-\Pr\left(\log X \leq -x \right)$. $\endgroup$ – JohnK Apr 10 '17 at 15:31
  • $\begingroup$ Okay, the calculations were pretty cool. Thanks! However... I've updated the question, since some conflicting results emerge. $\endgroup$ – dietervdf Apr 10 '17 at 16:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.