# Why is the quadratic approximation to the relative likelihood positive?

We can approximate the log likelihood at the real parameter value $$l(\theta)$$ with the MLE estimate $$l(\hat\theta)$$ using second order Taylor polynomials, like so:

$$l(\theta) - l(\hat\theta) \approx -\frac 1 2I(\hat\theta)(\theta - \hat\theta)^2$$

where $$I(\hat\theta)$$ is the observed Fisher information. It is always positive, as is the square term that follows. This implies that this difference in logs is negative. How can that be? This says that the log likelihood is higher under the MLE estimate than the true parameter value - shouldn't the truth be the highest attainable likelihood?

• Where is the first order tern? – user158565 Nov 2 '18 at 4:09
• It's expected value is zero so it's not there. – badmax Nov 2 '18 at 4:16
• Then should be $l(\theta)-\mathrm{E}(l(\hat\theta)) = -\frac12 ...$ – user158565 Nov 2 '18 at 4:18
• since $\hat\theta$ is the MLE, $\ell(\hat\theta)\ge\ell(\theta)$ – Xi'an Nov 2 '18 at 4:46

24.50 20.28 15.36 20.67 21.87 14.35 20.06

for simplicity, let's also assume we know $$\sigma=3$$, and look at the likelihood as a function of $$\mu$$.
The log likelihood will be maximized at $$\mu=\bar{x}$$ not at $$\mu=20$$; that's because $$\bar{x}$$ minimizes the sums of squared deviations from the observations you have, so it maximizes the likelihood for those data.