2
$\begingroup$

Let $X_i$ for $i = 1, ..., n$ be a random sample from the distribution $N(\mu, \mu^2 \tau)$ with unknown parameters $\mu \in (\infty, 0) \cup (0 ,\infty), \tau > 0$.

Find and justify the mle $\hat{\tau}$ and the limiting distribution of $\sqrt{n}(\hat{\tau} - \tau)$ as $n \rightarrow \infty$.

My Solution:

So from exponential theory the MLE's are moment-of-moments estimators, so $\hat{\mu} = \bar{X}$ and $\hat{\tau} = (1-1/n)S_X^2/(\bar{X})^2$. (Obviously after showing that this is a natural exponential family)

Now since the MLE contains the sample mean and sample standard deviation I could use LLN + CLT + Delta Method to find the limiting distribution but I know that the asymptotic property of the MLE is $\sqrt n(\hat{\tau} - \tau) \rightarrow N(0, \frac{1}{I(\tau)})$, so

$$l = \ln( f(x)) \propto -\frac12\log(\mu^2 \tau) - \frac{x^2}{2\mu^2 \tau} + \frac{x}{\mu \tau} - \frac{1}{2 \tau}$$

$$\frac{\partial l}{\partial \tau} = \frac{x^2}{2 \mu^2 \tau^2} - \frac{x}{\mu \tau^2} + \frac{1}{2 \tau^2} - \frac{1}{2 \tau}$$

$$\frac{\partial^2 l}{\partial \tau^2} = -\frac{x^2}{ \mu^2 \tau^3} + \frac{2x}{\mu \tau^3} - \frac{1}{ \tau^3} + \frac{1}{2 \tau^2}$$

$$-\frac{\partial^2 l}{\partial \tau^2} = +\frac{x^2}{ \mu^2 \tau^3} -\frac{2x}{\mu \tau^3} + \frac{1}{ \tau^3} - \frac{1}{2 \tau^2}$$

$$E\left(-\frac{\partial^2 l}{\partial \tau^2}\right) = \frac{\mu^2\tau + \mu^2}{ \mu^2 \tau^3} -\frac{2 \mu}{\mu \tau^3} + \frac{1}{ \tau^3} - \frac{1}{2 \tau^2}$$

$$E\left(-\frac{\partial^2 l}{\partial \tau^2}\right) = \frac{\tau}{2\tau^3}$$

$$\frac{1}{I(\tau)} = 2\tau^2.$$

However the solution given to us using the delta method is: $\sqrt{n} (\hat{\tau} - \tau) \approx N(0, 2\tau^2 + 4\tau^3)$

Am I going wrong somewhere conceptually or algebraically?

$\endgroup$
2
  • 2
    $\begingroup$ Why don't you compute the full information matrix? Your calculation appears to assume $\mu$ is known but you state it is not. $\endgroup$
    – whuber
    Nov 18, 2023 at 20:10
  • $\begingroup$ I was under the impression that since $\frac{1}{I(\tau)}$ does not depend on $\mu$ and $\mu \ne 0$ we can follow through despite whether $\mu$ is known or not. Perhaps it is a conceptual error on my part but would the "full information matrix" be pertaining to finding $\frac{\partial^2 l}{\partial \tau \partial \mu}$, $\frac{\partial^2 l}{\partial \mu \partial \tau}$, and $\frac{\partial^2 l}{\partial \mu^2}$? And if so, wouldn't that be $\sqrt{n} (\hat{\tau} - \tau) \rightarrow N(0, \frac{1}{I(\mu, \tau)})$ ? (Unless this is the actual formula for the asymptotic distribution of the MLE) $\endgroup$
    – Stats_Rock
    Nov 18, 2023 at 20:35

1 Answer 1

7
$\begingroup$

Whuber's comment is to the point, since it matters whether you estimate two parameters or one.

Algebraically, you are correct up to and including the computation of the expected value of the negative 2nd derivative. But if $\mu$ is also estimated, than the variance of $\tau$ is not the inverse (here just the reciprocal) of this expected value.

If $\mu$ is also estimated, then your MLE is a $2 \times 1$ vector, and in order to obtain the variance of $\tau$, you need to compute the full $2 \times 2$ (negative of the) Hessian of the log-likelihood and invert it properly. You will find that the answer given to you is correct. There is no need to use the Delta method.

Intuitively, the variance is higher due to the uncertainty in co-estimating also $\mu$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.