3
$\begingroup$

I've been reviewing questions from a statistics exam of the last year. There is a question with the probability density function below

$$\displaystyle f(x,\theta) = \frac 1{2\theta^3}x^2e^{-\frac x\theta}$$

where $\displaystyle0<x<\infty$ and $\displaystyle0<\theta<\infty$.

Question is as follows:

a) Find the maximum likelihood estimator for $\displaystyle\theta$.

b) Find the minimum variance estimator $\widehat{\theta}$ for the given pdf.

c) Is the maximum likelihood estimator obtained in a) efficient and consistent?

I have found the answer to a) as $\widehat\theta=\frac {1} {3n}\sum x_n$.

However, I couldn't find the Cramer-Rao lower bound to the end. I've shown the equation below

$$\displaystyle \frac {d^2\log(f(x,\theta))} {d\theta^2} = \frac 3 {\theta^2}-2\frac x {\theta^3}$$

I should take the expectation of this, but I'm not sure what should I replace $E[x]$ with. Should I just make it equal to $\displaystyle \theta$ and go further with the solution?

By the way, could you also hint me how can I find the variance of the maximum likelihood estimator above for c)? I'd appreciate it

Thanks in advance

$\endgroup$
  • 1
    $\begingroup$ It would help to begin by computing the expectation and variance of this distribution. You can do this readily from their definition or you can look up properties of Gamma distributions :-). $\endgroup$ – whuber Nov 30 '11 at 22:24
  • $\begingroup$ Thanks for the input, appreciated it. But I couldn't follow you there, what you mean by readily doing this by their definition? I'm not that advanced of a statistics student, and we haven't used gamma distributions until now. So is there any other way other than using gamma dist? $\endgroup$ – mala213 Nov 30 '11 at 22:54
  • 1
    $\begingroup$ The reference to Gammas was so you could Google them and view the answers on the Wikipedia page. :-) You're obviously skilled enough at Calculus to do the integrals and I can't imagine you would be learning about the CR bound and ML without having first learned basics like the definition of expectation and moments! Regardless, the most elementary approach is to integrate $x f(x,\theta)dx$ to obtain the expectation $\mu$ and then integrate $(x-\mu)^2 f(x,\theta)dx$ to get the variance. If you've learned about moment generating functions, that's an even easier calculation. $\endgroup$ – whuber Nov 30 '11 at 22:58
  • 1
    $\begingroup$ I have rolled back this question to its first version, mala. When you completely changed it, you invalidated all the comments and created a lot of confusion. If you decide to ask a different question and haven't yet gotten any answers, it's ok to delete your question and start a new one, but please don't make such wholesale changes once you have posted something. $\endgroup$ – whuber Dec 1 '11 at 15:50
  • 2
    $\begingroup$ Question 1: Is $E[x] = \theta$? Question 2: a) Look carefully at the expression for $\hat{\theta}$. Does it resemble some other expression you may be familiar with? If so, can you use what you know about that expression to help you answer the question? b) Reread whuber's hints. $\endgroup$ – jbowman Dec 1 '11 at 19:49
4
$\begingroup$

To close this one:
The density is recognized as a Gamma distribution with shape parameter $k=3$ and unknown scale parameter $\theta$, so we have $E(X) = 3\theta$ and $\operatorname {Var}(X) = 3\theta^2$ Given an i.i.d sample, the expected value and variance of the MLE is then

$$E(\hat \theta_{MLE}) = \theta,\;\;\operatorname {Var}(\hat \theta_{MLE}) = \frac 1 {3n}\theta^2$$

For consistency, we can use then the sufficient conditions: $\lim_{n\rightarrow}E(\hat \theta_{MLE})=\theta$, holds, and $\lim_{n\rightarrow}\operatorname {Var}(\hat \theta_{MLE})=0 \Rightarrow \lim_{n\rightarrow}\frac 1 {3n}\theta^2=0$, holds too, so the MLE is consistent.

The Central Limit Theorem Holds and so

$$\sqrt n(\hat \theta_{MLE}-\theta) \rightarrow_d \mathcal N(0, \frac 13\theta^2)$$

The Fisher Information is $$\mathcal I(\theta) = -E\left[\frac 3 {\theta^2}-2\frac X {\theta^3}\right] = -\frac 3 {\theta^2} + 2\frac {E(X)}{\theta^3} = -\frac 3 {\theta^2} + 2\frac {3\theta}{\theta^3} = \frac 3{\theta^2}$$

and so the MLE achieves the Cramer-Rao bound.

As for the question about minimum variance estimator, @cardinal's answer here
https://math.stackexchange.com/questions/28779/minimum-variance-unbiased-estimator-for-scale-parameter-of-a-certain-gamma-distr
is a complete and sufficient statistic for estimating the answer.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.