Proof that the log-likelihood is asymptotically quadratic I was reading this article, where the author says that 
Maximum Likelihood (ML) estimates are asymptotically normal if 
the log-likelihood is asymptotically quadratic.
I have heard or read other times about the likelihood being asymptotically
quadratic (under conditions), but I have never read any proof of this.
Is anybody aware of such a proof?
It would be great to see a proof that shows also the rate of convergence
$(\sqrt{n}?)$ of the log-likelihood to a quadratic function.
 A: DISCUSSION: Local asymptotic quadraticity of the log-likelihood is proven under the same set of assumptions that prove the asymptotic normality of the ML estimator -it is not a prerequisite. The proof of MLE's asymptotic normality, given this set of assumptions, can be much more straightforward and short-and I provide it at the end.  I combine two very different book sources, David Pollard's unpublished "Asymptopia" (ch. 2), and Hayashi's "Econometrics") (ch.7). I follow the notation of Hayashi. Moreover this is a "$n\rightarrow \infty$" kind of proof.
Assume a sample of size $n$ of i.i.d. observations coming from a distribution with density function $f_X(x;\theta_0)$ and use the following notation:
$$\ln f_X(x_i;\theta) = m(x_i;\theta),\;\; \frac {\partial m(x_i;\theta)}{\partial \theta}= s(x_i;\theta),\;\;\frac {\partial^2 m(x_i;\theta)}{\partial \theta\partial \theta}= H(x_i;\theta)$$
We have in turn, the log-likelihood, the score , and the Hessian, all related to observation $i$, not to the whole sample.
Notation should be understood as reflecting vector-matrices (for more than one unknown parameter).
Consider the average log-likelihood function (makes no difference in ML estimation, and strictly speaking, we should consider this likelihood in order for ML to be included in the M-estimators family)
$$\ell (\mathbf x;\theta) = \frac 1n \sum_{i=1}^nm(x_i;\theta)$$
Make the following assumptions that hold for a neighborhood of $\theta_0$, $\mathcal N(\theta_0)$:
[a] $\qquad m(x_i;\theta)$ is indeed twice differentiable, with second derivative continuous at $\theta_0$
[b] $\qquad$Its first derivative evaluated at $\theta_0$, $s(x_i;\theta_0)$ is square-integrable with respect to the probability measure involved (in other words, $E[s(x_i;\theta_0)s(x_i;\theta_0)']$ exists and is finite).
[c] $\qquad$ Its second derivative $H(x_i;\theta)$ is dominated by an integrable function in all $\mathcal N(\theta_0)$, (which means $E\left[\text {sup}_{\theta \in N(\theta_0)}  ||H(x_i;\theta)||\right] <\infty$
[d] $\qquad$ $E[H(x_i;\theta)]$ is not a singular matrix (so its inverse exists)
[e] $\qquad$ $\theta_0$ is an interior point of the parameter space $\Theta$
[f] $\qquad$ There exists $\hat \theta: \hat \theta\xrightarrow{p}\theta_0$.  
As Pollard remarks, the reason why we consider asymptotic normality of an estimator given that it is consistent, is that its asymptotic distribution would be of no use to us if the estimator was inconsistent. 
LOCAL QUADRATICITY to go to ASYMPTOTIC NORMALITY.
We note that local quadraticity is proven for the sample average log-joint density, not for the sample log-likelihood -i.e. we are one step before viewing the sample log joint density as a likelihood function of $\theta$. But we will continue to call it "log-likelihood".
In the above framework, the proof for local quadraticity essentially consists of nothing more than proving that the remainder from a 2nd order Taylor expansion of $m(x_i;\theta)$ around $\theta_0$ goes to zero asymptotically.
1st Step: Local Quadraticity
Assumption $[a]$ permits us to take this 2nd-order Taylor expansion and assumptions $[b]$ and $[c]$ permit to consider its expected value:
$$E[m(x_i;\theta)] = E[m(x_i;\theta_0)]+(\theta-\theta_0)E[s(x_i;\theta_0)]+\frac12(\theta-\theta_0)^2E[H(x_i;\theta_0)] + (\theta-\theta_0)^2E[R_2(x_i;\theta-\theta_0)]$$
where we have used the Peano form of the remainder term. Note that expectations relate to the $X$'s, the ML estimator has not yet been introduced.
Now, from a prior step, when proving the existence of a consistent estimator, by the "Identification Condition" we know that  $E[m(x_i;\theta)]$ is at a maximum when $\theta =\theta_0$. Then due to assumption $[e]$ ($\theta_0$ is an interior point), this implies that $E[s(x_i;\theta_0)]=0$. If $\theta_0$ was not interior, then the Taylor expansion would retain a possibly non-zero linear term in $\theta$, and we would not be able to guarantee quadraticity (and this is why when $\theta_0$ is at the boundary the asymtptotic properties change).
Then one goes to show that the remainder converges uniformly in probability to zero  (too much to write it down here), and we are left with
$$\ell (\mathbf x;\theta) = \frac 1n \sum_{i=1}^nm(x_i;\theta) \xrightarrow{p}E[m(x_i;\theta)] \xrightarrow{p} E[m(x_i;\theta_0)]+\frac12(\theta-\theta_0)^2E[H(x_i;\theta_0)] +o_p[(\theta-\theta_0)^2] $$
which is what is called "being locally asymptotically quadratic".
2nd step: Asymptotic Normality of the ML estimator
Keeping in mind the above results, we now take the 2nd-order Taylor expansion of the sample log-likelihood around $\theta_0$, manipulating the linear term,
$$\ell (x_i;\theta) = \ell (x_i;\theta_0) + \frac{(\theta-\theta_0)}{\sqrt n}\cdot\frac 1{\sqrt n}\sum_{i=1}^{n}s(x_i;\theta_0)+\frac12(\theta-\theta_0)^2\frac 1n\sum_{i=1}^{n}H(x_i;\theta_0) + R_n(x_i;(\theta-\theta_0))$$
Here too the remainder is shown to converge uniformly in probability to zero. If then we consider the derivative of the above with respect to $\theta$, and we set it equal to zero, we will obtain
$$\frac 1{\sqrt n}\cdot\frac 1{\sqrt n}\sum_{i=1}^{n}s(x_i;\theta_0)+(\hat \theta-\theta_0)\frac 1n\sum_{i=1}^{n}H(x_i;\theta_0)=0$$
where $\hat \theta$ is a "consistent root", which exists by assumption $[f]$.
Assumption $[d]$ permits to manipulate around and write
$${\sqrt n}(\hat \theta-\theta_0) = \left(-\frac 1n\sum_{i=1}^{n}H(x_i;\theta_0)\right)^{-1}\cdot\frac 1{\sqrt n}\sum_{i=1}^{n}s(x_i;\theta_0)$$
Assumption $[e]$ invoked previously gives us $E[s(x_i;\theta_0)]=0$. Assumption $[b]$ gives us that $E[s(x_i;\theta_0)s(x_i;\theta_0)']$ exists and is finite. Then the Central Limit Theorem holds and we have
$$\frac 1{\sqrt n}\sum_{i=1}^{n}s(x_i;\theta_0) \rightarrow_d N\left(0, E[s(x_i;\theta_0)s(x_i;\theta_0)']\right)$$
Finally, assumption $[c]$ guarantees that the inverse term is finite so 
$${\sqrt n}(\hat \theta-\theta_0)\rightarrow_d N\left(0, (E[H(x_i;\theta_0)])^{-1}E[s(x_i;\theta_0)s(x_i;\theta_0)'](E[H(x_i;\theta_0)]^{-1})'\right)$$
DIRECT PROOF OF ASYMPTOTIC NORMALITY
Under the same set of assumptions, using the Mean-Value Theorem, we take a mean-value expansion of $\frac {\partial \ell(x_i;\hat \theta)}{\partial \theta}$ at $\theta_0$ and we obtain 
$$\frac {\partial \ell(x_i;\hat \theta)}{\partial \theta}=\frac 1n\sum_{i=1}^{n}s(x_i;\theta_0)+(\hat \theta-\theta_0)\frac 1n\sum_{i=1}^{n}H(x_i;\bar \theta)=0$$
where $\bar \theta$ is a mean value between $\hat \theta$ and $\theta_0$. The assumptions permit us to write
$${\sqrt n}(\hat \theta-\theta_0) = \left(-\frac 1n\sum_{i=1}^{n}H(x_i;\bar \theta)\right)^{-1}\cdot\frac 1{\sqrt n}\sum_{i=1}^{n}s(x_i;\theta_0)$$
$\bar \theta$ is sandwiched between $\hat \theta$ and $\theta_0$ and it is also  consistent for $\theta$. This and the assumptions, lead us to the same final result as before.
