1
$\begingroup$

I am trying to prove the consistency of MLE for a beta distribution. The problem now reduces to the following:

Assume $Y=\frac n X$ and $X$ ~ Gamma(n,$\frac 1 \theta$), prove that $Y$ converges to $\theta$ in probability when $n$ goes to infinity.

I set up the proof by the definition of convergence in probability but cannot prove it.

$\endgroup$
12
  • $\begingroup$ What definition did you use? What prevents you from carrying out the proof? $\endgroup$
    – whuber
    Jan 31 '16 at 18:23
  • $\begingroup$ The limit of $\rm{P}(|Y-\theta|<\epsilon)$ is 1 for any $\epsilon > 0$ when $n$ goes to infinity. Basically, this probability can be calculated as the difference of CDF of a gamma distribution, but it is hard for me to find a closed form of that CDF when integrate from $\frac n {\theta + \epsilon}$ to $\frac n {\theta - \epsilon}$. $\endgroup$
    – Sheldon
    Jan 31 '16 at 18:25
  • $\begingroup$ You might want to try to approximate that probability with a normal probability, which in the limit is exact of course. $\endgroup$
    – JohnK
    Jan 31 '16 at 18:42
  • $\begingroup$ @JohnK Thanks for your suggestion, I know that Gamma can be approximated by normal when $n$ is large. But I am not comfortable with this approximation because this is not the classical approach of proof for convergence in prob. $\endgroup$
    – Sheldon
    Jan 31 '16 at 18:51
  • 2
    $\begingroup$ Chebyshev's inequality does not require that the constant is the mean. You can prove that $$ E\left(Y - c \right)^2 \geq a^2 P\left[ | Y - c | \geq a \right] $$ for any constants $a>0$ and $c$. $\endgroup$
    – JohnK
    Jan 31 '16 at 19:17
2
$\begingroup$

This is also a direct consequence of the Law of Large numbers: if $X_n\sim\mathcal{G}(n,\theta)$, $X$ can be written as $$X_n=Y_1+\cdots+Y_n\qquad Y_i\stackrel{\text{iid}}{\sim}\mathcal{E}(\theta)$$ Therefore, $$\dfrac{X_n}{n}=\dfrac{\sum_{i=1}^n Y_i}{n}\stackrel{\text{a.s.}}{\longrightarrow}\mathbb{E}[Y_1]=\dfrac{1}{\theta}$$ and thus $$\dfrac{n}{X_n}\stackrel{\text{a.s.}}{\longrightarrow}\theta$$

$\endgroup$
1
$\begingroup$

The answer is using Chebyshev's Inequality to prove that: $$\rm{P} (|Y-\theta| \geq \epsilon) \leq \frac 1 {\epsilon^2} E[(Y-\theta)^2]$$ Since Y is equal to $\frac n X$ where $X$ is a Gamma distributed random variable ($X$ ~ Gamma ($n$,$\frac 1 \theta$)), one can calculate that: $$E[(Y-\theta)^2]=\frac {n^2\theta^2} {(n-1)(n-2)} - \frac {2n\theta^2} {n-1} + \theta^2$$ which goes to 0 when $n$ goes to infinity. This demonstrates that $$\rm{P} (|Y-\theta| \geq \epsilon)=0$$ when $n$ goes to infinity. Therefore, Y is a consistent estimator of $\theta$ (Y converge to $\theta$ in probability).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.