# Why is the quadratic approximation to the relative likelihood positive?

We can approximate the log likelihood at the real parameter value $$l(\theta)$$ with the MLE estimate $$l(\hat\theta)$$ using second order Taylor polynomials, like so:

$$l(\theta) - l(\hat\theta) \approx -\frac 1 2I(\hat\theta)(\theta - \hat\theta)^2$$

where $$I(\hat\theta)$$ is the observed Fisher information. It is always positive, as is the square term that follows. This implies that this difference in logs is negative. How can that be? This says that the log likelihood is higher under the MLE estimate than the true parameter value - shouldn't the truth be the highest attainable likelihood?

• Where is the first order tern? Commented Nov 2, 2018 at 4:09
• It's expected value is zero so it's not there. Commented Nov 2, 2018 at 4:16
• Then should be $l(\theta)-\mathrm{E}(l(\hat\theta)) = -\frac12 ...$ Commented Nov 2, 2018 at 4:18
• since $\hat\theta$ is the MLE, $\ell(\hat\theta)\ge\ell(\theta)$ Commented Nov 2, 2018 at 4:46

24.50 20.28 15.36 20.67 21.87 14.35 20.06

for simplicity, let's also assume we know $$\sigma=3$$, and look at the likelihood as a function of $$\mu$$.
The log likelihood will be maximized at $$\mu=\bar{x}$$ not at $$\mu=20$$; that's because $$\bar{x}$$ minimizes the sums of squared deviations from the observations you have, so it maximizes the likelihood for those data.