I got a question when I tried to prove the distribution of the Maximum likelihood test statistics.
The test is $H_0:\theta=\theta_0$ $H_1:\theta\neq\theta_0$
The statistics is $MLR=\frac{\sum f(x_i|\hat{\theta})}{\sum f(x_i|\theta_0)}$
So I want to show $2 \times log(MLR)$, which is $2 \times (l(\hat{\theta})-l(\theta_0)) \sim \chi^2(1)$ under the null hypothesis.
Here is a proof (page 74):http://www.stats.ox.ac.uk/~dlunn/b8_02/b8pdf_8.pdf
I understand how he proved it. The first step is the Taylor expansion:
$l(\theta_0)=l(\hat{\theta})+\frac{1}{2}(\theta_0-\hat{\theta})^2l''(\hat{\theta})$
Then
$2(l(\hat{\theta})-l(\theta_0))=(\theta_0-\hat{\theta})^2(-l''(\hat{\theta}))$
We can show that the right-hand side follows $\chi^2(1)$, then we proved it.
However, if we use Taylor expansion at the point $\theta_0$, we have:
$l(\hat{\theta})=l(\theta_0)+\frac{1}{2}(\theta_0-\hat{\theta})^2l''(\theta_0)$
Then
$2 (l(\hat{\theta})-l(\theta_0))=(\theta_0-\hat{\theta})^2l''(\theta_0)$
The left-hand sides are exactly same for the above two, but for the right-hand sides, one is $-l''(\hat{\theta})$ and the other is $l''(\theta_0)$. As far as I know, $l''(\hat{\theta})$ and $l''(\theta_0)$ will converge to the same value when n goes to infinity. Therefore, $-l''(\hat{\theta})$ and $l''(\theta_0)$ have opposite signs, so:
$2(l(\hat{\theta})-l(\theta_0))=(\theta_0-\hat{\theta})^2(-l''(\hat{\theta}))$ will converge to $\chi^2(1)$
And
$2(l(\hat{\theta})-l(\theta_0))=(\theta_0-\hat{\theta})^2l''(\theta_0)$ will converge to $- \chi^2(1)$
I know the second one must be wrong because $l(\hat{\theta})-l(\theta_0)\geq 0$. I just cannot find where my mistake is. Any idea?
