Lemma 3 of the note on the consistencty of the maximum likelihood estimate by A.Wald

Question

Can someone explain to me how Lemma 3 in this article follows from (13) and (14) and the fact that the functions are decreasing?

Below is my transcription of the Lemma.

Let $X_1, X_2,...$ - i.i.d. random variables, with common distribution function $F(x,\theta)$ known, except for parameters $\theta = (\theta^1, \ldots, \theta^k)$. Let $f(x, \theta)$ denote density/probability of x.

Let's define:

1 $\mathbf{\varphi(x,r) = \sup_{\theta} f(x, \theta)}; \ \ \ \ |\theta| > r, \ r>0$

2 $ \mathbf{\varphi^*(x,r)} = \left\{ \begin{array}{ll} \mathbf{\varphi^*(x, r) = \varphi(x, r)} & \textrm{when $ \varphi(x, r)>1$}\\ \mathbf{\varphi^*(x, r) = 1} & \textrm{otherwise} \end{array} \right.$

Let's assume:

1. For sufficiently large r, the expected value: \begin{equation} E\log\varphi ^*(x, r) = \int_{-\infty}^{\infty} \log \varphi^*(x,r) dF(x, \theta_0) \end{equation} is finite where $\theta_0$ denotes the true parameter point;

2. If $\lim\limits_{i \rightarrow \infty} |\theta_i| = \infty$, then $\lim\limits_{i \rightarrow \infty} f(x, \theta_i) = 0$, for any x, except perhaps on a fixed set whose probability measure is zero according to the probablility distribution corresponding to the true parameter point $\theta_0$;

Lemma 3

The equation: \begin{equation} \lim\limits_{r=\infty}Elog \varphi(X,r) = -\infty \end{equation} holds

Proof:

It follows from Assumption 2 that: \begin{equation} \lim\limits_{r \to \infty}\log \varphi(x, r) = -\infty \ \ \ \ \ (1) \end{equation}
for any x (except perhaps on a set of probability zero). Since according to Assumption 1 \begin{equation} E\log \varphi^*(X, r) < \infty \end{equation} and since $\log \varphi(x,r) - \log \varphi^*(x,r)$ and $\log \varphi^*(x, r)$ are decreasing functions of r, so Lemma 3 easily follows from (1).

Could someone explain to me why:

1. Function $\log \varphi(x,r) - \log \varphi^*(x,r)$ is decreasing when r is increasing

2. Functions $\log \varphi(x,r) - \log \varphi^*(x,r)$ and $\log \varphi^*(x, r)$ need to be decreasing functions of r to prove the Lemma 3

3. The Lemma 3 follows from (1) and the fact that the functions $\log \varphi(x,r) - \log \varphi^*(x,r)$ and $\log \varphi^*(x, r)$ are decreasing

Answer 1

Note that "decreasing" is used here to include strictly decreasing and non-increasing.

1. If we fix $x$, then as $r$ increases we have a smaller space over which to find the supremum over $\theta$, so $\phi (x,r)$ is a decreasing function in $r$. Since the logarithm is monotonic, then $\log \phi (x,r)$ is also a decreasing function in $r$.

Since $\phi (x,r)$ is decreasing in $r$, then $\phi^* (x,r)$ is decreasing as long as $\phi (x,r)>1$. When $\phi (x,r) \le 1$, then $\phi^* (x,r)$ becomes equal to 1, and so is trivially decreasing thereafter in $r$. This with the monotonicity of the logarithm gives us that $\log \phi^* (x,r)$ is decreasing in $r$.

Now, looking at $$\log \phi (x,r) - \log\phi^* (x,r) = \log \frac {\phi (x,r)}{\phi^* (x,r)},$$ we see that the ratio inside the logarithm is given by $$\frac {\phi (x,r)}{\phi^* (x,r)} = \left\{ \begin{array}{rl} 1 & \phi (x,r) > 1 \\ \phi (x,r) & \phi (x,r) \le 1 \end{array} \right.. $$ Taking a look at the right-hand side, we see that since $\phi (x,r)$ is decreasing, that the ratio is decreasing. Yet again we use the monotonicity of the logarithm.

2. It is not clear whether these are necessary conditions, yet.

3. We want to determine the limit as $r\rightarrow\infty$ of $E \log\phi (x,r)$. We can rewrite this limit as $$ \begin{align} \lim_{r\rightarrow\infty} E\log\phi (x,r) & = \lim_{r\rightarrow\infty} E [\log\phi (x,r) - \log\phi^* (x,r) + \log\phi^* (x,r)] \\ &= \lim_{r\rightarrow\infty} E [\log\phi (x,r) - \log\phi^* (x,r)] + \lim_{r\rightarrow\infty} E \log\phi^* (x,r). \end{align} $$ The second term of the second line is finite by Assumption 2 of the paper.

By assumption 5 of the paper we have $$\lim_{r\rightarrow\infty} \log\phi (x,r) = -\infty.$$ Therefore, for all $x$ we have $$ \lim_{r\rightarrow\infty} (\log\phi (x,r) - \log\phi^* (x,r)) = -\infty, $$ by looking at the logarithm of the ratio shown above.

This still needs clarification, but the last step is to prove that the limit of the expectation (i.e., the first term of the second line) diverges to $-\infty$. That, coupled with the boundedness of the other term, proves the lemma. The last step looks a bit like some form of the monotone convergence theorem might somehow apply --- or perhaps it is more simple?