# Proving a result in maximum likelihood theory: accuracy of the quadratic approx. to loglikelihood

I need help proving a result shown in a paper. I am reading Assessing the Quadratic Approximation to the Log Likelihood Function in Nonnormal Linear Models by Salomon Minkin.

The paper defines several concepts that are used in the argument, so I'll state those here:

## Definitions

Let $$\beta \in \mathbb{R}^m$$.

Let $$L(\beta;x)$$ be a a log-likelihood function and $$\hat{\beta}$$ be the maximum likelihood estimator (MLE) of the parameter $$\beta$$ given data $$x$$ ($$x$$ is a vector of observations)

Let $$Q(\beta;\hat{\beta},x)$$ be the quadratic approximation to $$L(\beta;x)$$ around the $$\hat{\beta}$$ (i.e., the second degree Taylor approximation about $$\hat{\beta}$$).

The argument uses the "drop" in the loglikelihood functions as you move away from their common maximizer at $$\hat{\beta}$$

Let $$D_L(\beta):=D_L(\beta;\hat{\beta},x)=L(\hat{\beta};x) - L(\beta;x)$$ be the drop in the $$L$$ at point $$\beta$$.

Similarly, $$D_Q(\beta):=D_Q(\beta;\hat{\beta},x)=Q(\hat{\beta};\hat{\beta},x) - Q(\beta;\hat{\beta},x)$$ be the drop in the quadratic approximation to $$L$$ at $$\beta$$.

## Proof Plan

The author then uses $$D_L,D_Q$$ in the main part of the argument, reproduced below:

To me it's not clear/obvious that $$\beta\in R_L(a_0-d) \implies \beta \in R_Q(a_0) \;\;\forall a_0 \leq a$$ so I tried to prove this to myself.

Let's start with $$a_0=a$$. Here are the series of implications I am trying to prove (there may be a more straightforward approach). The first implication (marked with $$(*)$$) is the one I'm not sure about, but if I can prove it the rest of the proof here is pretty straightforward. I basically assumed it was true and continued on the proof, now coming back to shore up $$(*)$$.

$$(*): \beta \in R_L(a-d) \implies \beta \in R_A(d)$$

$$(1): (*) \implies D_Q(\beta) < D_L + d$$

$$(2): \beta \in R_L(a-d) \implies D_L(\beta) \leq a - d$$

$$(3): (2) \wedge (1) \implies D_Q(\beta) < D_L + d \leq (a-d) + d \implies D_Q(\beta) < a$$

$$(4): (3) \implies \beta \in R_Q(a) \implies R_L(a-d) \subset R_Q(a)$$

So, assuming $$(*)$$ we've shown $$\beta \in R_L(a-d) \implies \beta \in R_Q(a)$$.

The generalization to $$a_0\leq a$$ is also straightforward due the definition of $$R_Q$$. For any decreasing sequence of values $$s_1>s_2>s_3...>s_n$$ defines a decreasing sequence of sets in $$\mathbb{R}^{m}$$ (ellipsoids) $$R_Q(s_1)\supset R_Q(s_2)\supset R_Q(s_3)...\supset R_Q(s_n)$$

Substituting $$a_0 < a$$ for $$a$$ into $$(*)$$ and proceeding with $$(1)-(4)$$ gets us the general result stated in the paper:

$$\beta \in R_L(a_0-d) \implies D_Q(\beta) < a_0 \implies \beta \in R_Q(a_0) \implies R_L(a_0-d) \subset R_Q(a_0) \;\;\forall a_0 \leq a$$ $$\square$$

## Proving $$(*)$$

The jumping off point of my proof is $$(*)$$, which I am not sure I can rigorously prove.

I feel it needs to use the fact that $$R_Q(a) \subset R_A(d)$$ and that the regions $$R_L(z)$$ are connected (as mentioned in the above passage).

My attempt

$$R_Q(a) \subset R_A(d) \implies |D_Q(\beta)-D_L(\beta)| < d\;\; \forall \beta \in R_Q(a)$$ by the definition of $$R_A(d)$$

To show $$R_L(a-d) \in R_A(d)$$, there are two cases:

Case 1:$$R_L(a) \setminus R_Q(a) = \emptyset$$

$$R_L(a) \setminus R_Q(a) = \emptyset \implies R_L(a) \subset R_Q(a) \implies R_L(a) \subset R_A(d)$$.

By the definition of $$R_L(z)$$, $$R_L(a-d) \subset R_L(a) \implies R_L(a-d) \subset R_A(d)$$.

Case 2:$$R_L(a) \setminus R_Q(a) \neq \emptyset$$

Let $$G = R_L(a) \setminus R_Q(a)$$

In this case, we know that $$D_L(g) < D_Q(g)\;\;\forall g \in G$$:

Proof: Assume $$g \in G: D_L(g)-D_Q(g) = k > 0$$. This means that $$D_L(g) = a+k$$ which contradicts the fact that $$g \in R_L(a)\;\;\;\square$$

Since $$D_L(g)< D_Q(g)$$ over $$G$$, and also that $$R_Q(a) \in R_A(d)$$, then $$\exists c. By definition, $$R_L(a-d) \subset R_L(a-c) \subset R_Q(a) \subset R_A(d)\;\; \square$$

Turns out this appears to simultaneously prove the overall point I was trying to understand.

Is this correct? Am I missing something?