1
$\begingroup$

I got a question when I tried to prove the distribution of the Maximum likelihood test statistics.

The test is $H_0:\theta=\theta_0$ $H_1:\theta\neq\theta_0$

The statistics is $MLR=\frac{\sum f(x_i|\hat{\theta})}{\sum f(x_i|\theta_0)}$

So I want to show $2 \times log(MLR)$, which is $2 \times (l(\hat{\theta})-l(\theta_0)) \sim \chi^2(1)$ under the null hypothesis.

Here is a proof (page 74):http://www.stats.ox.ac.uk/~dlunn/b8_02/b8pdf_8.pdf

I understand how he proved it. The first step is the Taylor expansion:

$l(\theta_0)=l(\hat{\theta})+\frac{1}{2}(\theta_0-\hat{\theta})^2l''(\hat{\theta})$

Then

$2(l(\hat{\theta})-l(\theta_0))=(\theta_0-\hat{\theta})^2(-l''(\hat{\theta}))$

We can show that the right-hand side follows $\chi^2(1)$, then we proved it.

However, if we use Taylor expansion at the point $\theta_0$, we have:

$l(\hat{\theta})=l(\theta_0)+\frac{1}{2}(\theta_0-\hat{\theta})^2l''(\theta_0)$

Then

$2 (l(\hat{\theta})-l(\theta_0))=(\theta_0-\hat{\theta})^2l''(\theta_0)$

The left-hand sides are exactly same for the above two, but for the right-hand sides, one is $-l''(\hat{\theta})$ and the other is $l''(\theta_0)$. As far as I know, $l''(\hat{\theta})$ and $l''(\theta_0)$ will converge to the same value when n goes to infinity. Therefore, $-l''(\hat{\theta})$ and $l''(\theta_0)$ have opposite signs, so:

$2(l(\hat{\theta})-l(\theta_0))=(\theta_0-\hat{\theta})^2(-l''(\hat{\theta}))$ will converge to $\chi^2(1)$

And

$2(l(\hat{\theta})-l(\theta_0))=(\theta_0-\hat{\theta})^2l''(\theta_0)$ will converge to $- \chi^2(1)$

I know the second one must be wrong because $l(\hat{\theta})-l(\theta_0)\geq 0$. I just cannot find where my mistake is. Any idea?

$\endgroup$
1
$\begingroup$

In your first formula, the term $\ell'(\hat\theta)$ is zero because at the MLE the derivative of the log likelihood is zero. But the first derivative of the likelihood need not be zero at the true value of the parameter, so your Taylor expansion around $\theta_0$ appears to be incorrect.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.