5
$\begingroup$

From 'Modern Mathematical Statistics with Applications' (Devore and Beck) pg 377

Let $X_1, X_2 \ldots$ be a random sample from the disbribution $f(x,\theta) = \theta x^{\theta-1}$ for $x\in {]0,1[}$ and $\theta >0$.

The MLE is given by $\hat \theta = \dfrac{-n}{\sum_i\ln X_i}$.

It also shows how $$\sqrt{n}(\hat \theta - \theta) \stackrel{\text{D}}{\rightarrow}N(0,\theta)$$

Using the fact that the MLE is consistent and the CLT.

I wonder if this estimator is also unbiased, I want to show how $E[\hat \theta] = \theta $. Any ideas (no full solutions please) on how to (dis)prove this.

Here are some things I've tried:

  • Notice how $E[\ln X] = \frac{-1}{\theta}$ or $E[\sum_i \ln X_i]= \dfrac{-n}{\theta}$

  • Calculating the expected value seems cumbersome:

$$E\left[\frac{-n}{\sum_i \ln X_i}\right] = \int_0^1\ldots \int_0^1 \frac{-n}{\sum_i \ln x_i} \cdot \theta^n x_1^{\theta-1}x_2^{\theta-1}\ldots x_n^{\theta-1}\operatorname d x_1\ldots \operatorname dx_n$$

Edit after the responses from JohnK and Alecos Papadopoulos

  • Direct calculation: Okay, this is pretty cool. I worked it out and found how $\sum_i -\ln X_i = \sum Y_i \stackrel{\text{d}}{=} \Gamma(n,\theta)$ (through the hints supplied byJohnK), then I immedialty used LOTUS and found $E[\hat \theta] = \dfrac{n}{n-1}\theta > \theta$.

  • Jensen's reasoning: I guess I should use Jensen's inequality as follows, since $E\left[\sum_i \ln X_i\right] = \dfrac{-n}{\theta}$ and because $\sum_i \ln X_i \in {]-\infty, 0[}$ I should look at the left part of the function $g:x\mapsto \dfrac{-n}{x}$ which is stictly convex. Jensen's concludes: $$E[g(X)] > g(E[X])$$ this would imply here: $$E[\hat \theta] > \theta$$

Improve this question
$\endgroup$
7
  • $\begingroup$ What is wrong with the first remark? - woops, noticed the typo :) $\endgroup$
    – dietervdf
    Commented Apr 10, 2017 at 14:48
  • $\begingroup$ @AlecosPapadopoulos: Okay, using Jensen's results in $E[\hat \theta] \leqslant \theta$ (since $g: x\mapsto \frac{-n}{x}$ is concave in $]0,1[$), but this doesn't imply unbiasedness? $\endgroup$
    – dietervdf
    Commented Apr 10, 2017 at 14:55
  • $\begingroup$ Strict inequality $\endgroup$
    – Alecos Papadopoulos
    Commented Apr 10, 2017 at 14:55
  • $\begingroup$ Isn't Jensen's inequality always a weak inequality? $\endgroup$
    – dietervdf
    Commented Apr 10, 2017 at 15:07
  • 1
    $\begingroup$ Since the MLE of a transform is the transform of the MLE, the MLE is almost never unbiased! $\endgroup$
    – Xi'an
    Commented Nov 7, 2018 at 10:06

1 Answer 1

Reset to default
5
$\begingroup$

Here is a sequence of steps that will help:

-Find the distribution of $Y=-\log X$

-Find the distribution of $\sum_{i=1}^n Y_i = - \sum_{i=1}^n \log X_i$

-Lastly, evaluate the expectation of $Z = \frac{1}{\bar{Y}}$. You can do this directly using the distribution of the previous step and LOTUS or by first finding the distribution of $Z$.

Hint: the gamma family is a truly large family of distributions.

Improve this answer
$\endgroup$
8
  • $\begingroup$ - Since $X = e^{-Y}$ one finds $F_Y(y \theta) = e^{-\theta y}$ or $f_Y(y, \theta) = -\theta e^{-\theta y}$ where $y\in {]0, +\infty[}$, can you expand on the distribution of that sum $\sum_i Y_i$? $\endgroup$
    – dietervdf
    Commented Apr 10, 2017 at 15:10
  • $\begingroup$ @dietervdf Check your derivation again, that minus sign shouldn't be there. Then for the distribution of the sum, you need to use the additivity property of the gamma family of distributions. If I say more, I will give it away. $\endgroup$
    – JohnK
    Commented Apr 10, 2017 at 15:16
  • $\begingroup$ I think I see where you are going, thanks for the hint! Just one thing, why shouldn't the minus be there? My derivation is as follows: $F_X(x,\theta) = x^\theta$ (with $x\in {]0,1[}$, then using the mapping $X = e^{-Y}$ one gets the cdf of $Y$, as $F_Y(y,\theta) = e^{-\theta y}$ which implies $f_Y(y,\theta) = -\theta e^{-\theta y}$ $\endgroup$
    – dietervdf
    Commented Apr 10, 2017 at 15:26
  • 1
    $\begingroup$ @dietervdf You you need to take the complement of the CDF as you have a minus there. $\Pr \left( - \log X \leq x \right) = 1-\Pr\left(\log X \leq -x \right)$. $\endgroup$
    – JohnK
    Commented Apr 10, 2017 at 15:31
  • $\begingroup$ Okay, the calculations were pretty cool. Thanks! However... I've updated the question, since some conflicting results emerge. $\endgroup$
    – dietervdf
    Commented Apr 10, 2017 at 16:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.