Let $y_1, \dots,y_n$ be i.i.d. random variables from $Y$ with density $$p(y;\theta)=\theta(1-y)^{\theta-1},\ \theta > 0,\ y \in(0,1)$$

and $T=-\ln(1-Y)\sim Exp(\theta)$ with $\theta$ rate parameter.

The log-likelihood is $l(\theta)=n \ln(\theta)+\sum\ln(1-y)$ and the MLE is $\hat \theta=-\frac{n}{\sum \ln(1-y)}$.

I need to prove the (weak) consistency of $\hat \theta$. A sufficient condition is
$$\begin{cases} \lim_{n \to \infty}\mathbb{E}(\hat \theta)=\theta \\ \lim_{n \to \infty}\mathbb{Var}(\hat \theta)=0 \end{cases}$$

Based on MLE invariance I could write $\hat \theta=\frac{n}{\sum t}$, so $$\lim_{n \to \infty}\mathbb{E}(\hat \theta)=\lim_{n \to \infty}\mathbb{E}\left(\frac{n}{\sum t}\right)=\lim_{n \to \infty}\left(n\frac{1}{n\frac{1}{\theta}}\right)=\theta$$

and $$\mathbb{Var}(\hat \theta)=\mathbb{Var}\left(\frac{n}{\sum t}\right)=n^2\mathbb{Var}\left(\frac{1}{\sum\mathbb{Var}(t)}\right)=n^2\frac{1}{\frac{n}{\theta^2}}=n\theta^2$$

What am I doing wrong?

Edit with solution

My solution: if $\sum t_i \sim \Gamma(n,\theta)$ then $\frac{1}{\sum t_i}\sim IG(n,\theta)$. Now, the variance of $IG$ should be $\mathbb{Var}(IG(n,\theta))=\frac{\theta^{2}}{(n-1)^2(n-2)}$ so $\lim_{n \to \infty}n^2\frac{\theta^{2}}{(n-1)^2(n-2)}=0$ and this prove the consistency of $\hat \theta$.

  • $\begingroup$ Your estimator is the inverse of an average. There is no way the variance of that estimator is ever going to zero. The CLT works because variances of averages fall the more information is available. Your MLE invariance is therefore wrong. $\endgroup$ – luchonacho Aug 26 '16 at 17:29

The error you have made is that $$\text{Var}\left( \dfrac{1}{\sum t_i} \right) \ne \dfrac{1}{\sum \text{Var}(t_i)}.$$

Instead you need to find the distribution of $\sum t_i$, where $t_i \sim Exp(\theta)$. Then find the distribution of $1/\sum t_i$. Hint, you should end up with an Inverse Gamma distribution. Find the variance of that distribution, and plug it in.

  • $\begingroup$ Hi, I try to put everything together. If $\sum t_i \sim \Gamma(n,\theta)$ then $\frac{1}{\sum t_i}\sim IG(n,\theta^{-1})$. Now, the variance of $IG$ should be $\mathbb{Var}(IG(n,\theta^{-1}))=\frac{\theta^{-2}}{(n-1)^2(n-2)}$ so $\lim_{n \to \infty}n^2\frac{\theta^{-2}}{(n-1)^2(n-2)}=0$ and this prove the consistency of $\hat \theta$. $\endgroup$ – Paul Aug 27 '16 at 13:53
  • $\begingroup$ Good. You did make a mistake though I think. It is written that $\theta$ is the rate parameter. Thus $\sum t_i \sim \Gamma(n, \theta)$ where $\theta$ is the rate parameter. For the rate parameter, $1/\sum t_i \sim IG(n, \theta)$, not $\theta^{-1}$. See my answer here. $\endgroup$ – Greenparker Aug 27 '16 at 13:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.