3
$\begingroup$

Let $X_1,\dots, X_n$ be iid sample from $X\sim f(x;\theta)=(\theta+1)x^\theta$ for $\theta>0$. Find the MLE say $\hat{\theta}$ and the asymptotic distribution of $\hat{\theta}$.


My work: The log-likelihood function is that $$ \ell(\theta)=n\log(\theta+1)+\sum \theta\log X_i. $$

The MLE is that $$ \hat{\theta}=-\left(\frac{n}{\sum\log X_i}+1\right). $$

Now, the asymptotic distribution of $\hat{\theta}$ should be $$ \sqrt{n}(\hat{\theta}-\theta_0)\to N(0, I^{-1}(\theta_0)) $$ where $$I(\theta)=E\left[\left(\frac{\partial \ell(\theta)}{\partial \theta}\right)^2\right].$$

Here $\hat{\theta}\to \theta$ in probability.

So the result is that $$\sqrt{n}(\hat{\theta}-\theta)\to N\left(0, \frac{(\theta+1)^2}{n}\right)$$?

$\endgroup$
5
  • $\begingroup$ closevoters, the OP is showing efforts at solving the problem. $\endgroup$
    – utobi
    Mar 30, 2023 at 15:07
  • $\begingroup$ @utobi they are probably voting to close because OP didn't have the tag, which implied they didn't read its wiki. The current consensus is to not add it for the OP $\endgroup$
    – Firebug
    Mar 30, 2023 at 15:15
  • 1
    $\begingroup$ Thanks for the clarification @Firebug. $\endgroup$
    – utobi
    Mar 31, 2023 at 5:02
  • 1
    $\begingroup$ A curiousity here is that $1/(\hat\theta+1)$ follows a Gamma distribution exactly with shape parameter $n$ and rate parameter $\theta+1$. So the distribution of $\hat\theta$ is known exactly and hence, except as an undergraduate math stat exercise, computing an asymptotic distibution using Fisher information is unnecessary. $\endgroup$ Mar 31, 2023 at 5:20
  • $\begingroup$ Please specify the range of $X$; presumably $X\in (0,1)$. $\endgroup$
    – utobi
    Mar 31, 2023 at 7:04

2 Answers 2

6
$\begingroup$

Your work is almost fine and a bit incomplete. To complete the answer you'll have to compute the Fisher information

\begin{align} I_1(\theta) & = -E\left(\frac{d^2\ell(\theta;X_1)}{d\theta d\theta}\right) \end{align}

where $\ell(\theta;X_1) = \log f(X_1;\theta)$, is the log-likelihood for a single observation, here chosen to be the first. You can also use its equivalent expression based on squared the score function, though that may lead to slightly longer computations.

Remark You can also use the asymptotically equivalent version based on the observed information $J(\theta) = -\frac{d^2\ell(\theta)}{d\theta d\theta}$. The rationale for this choice is that, under broad regularity conditions, $J(\theta)/n$ converges in probability to $I_1(\theta)$ for $n\to\infty$.

$\endgroup$
2
  • $\begingroup$ So the result is that $\sqrt{n}(\hat{\theta}-\theta)\to N(0, \frac{(\theta+1)^2}{n})$? $\endgroup$
    – Hermi
    Mar 31, 2023 at 4:31
  • $\begingroup$ Y, but Without the n on the r.h.s $\endgroup$
    – utobi
    Mar 31, 2023 at 4:56
2
$\begingroup$

So far so good. Just one thing you need to be careful with:

In your expression for the asymptotic distribution of $\sqrt{n}(\hat{\theta}-\theta_0)$, the term $\mathcal{I}(\theta)$ is the information contained in a single observation.

I prefer the notation $\mathcal{I}_{X_1}(\theta)$ for this quantity, and $\mathcal{I}_{\mathbf{X}}(\theta) = n I_{X_1}(\theta)$ for the information contained in a random sample of size $n$.

Also, remember that there is an alternative formula for the Fisher information, $$ \mathcal{I}_{\mathbf{X}}(\theta)= -\mathbb{E}\left(\frac{\partial^2}{\partial \theta^2} \ell(\theta)\right) $$ This is much easier to evaluate in this situation.

$\endgroup$
2
  • $\begingroup$ But what is $\theta_0$? $\endgroup$
    – Hermi
    Mar 31, 2023 at 4:32
  • $\begingroup$ @Hermi, $\theta_0$ or $\theta\$ denotes the true parameter value. $\endgroup$
    – utobi
    Mar 31, 2023 at 4:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.