Consistency of M-estimator?

Question

The reported results can be found in van der Vaart's "Asymptotic Statistics".

---

I am having some difficulties to understand the logic behind the following proof provided by the author:

Theorem $5.7.$ Let $M_n$ be random functions and let $M$ be a fixed function of $\theta$ such that $\forall \epsilon>0$

$$\sup_{\theta} \vert M_n(\theta)-M(\theta)\vert\to 0, $$ $$\sup_{\theta:d(\theta,\theta_0)\geq \epsilon} M(\theta)<M(\theta_0)$$ Then any sequence of estimators $\hat{\theta}_n$ with $M_n(\hat{\theta}_n)\geq M_n(\theta_0)-o_P(1)$ converges in probability to $\theta_0$.

(all functions live in a space topologized by convergence in probability so all limit's should be understood as limit in probability).

Proof:

Assume $\hat{\theta}_n$ satisfies the conditions, so we have $$M_n(\hat{\theta}_n)\geq M_n(\theta_0)-o_P(1)$$

Because uniform convergence of $M_n$ to $M$ implies convergence of $M_n(\theta_0)$ to $M(\theta_0)$ $\color{red}{\text{ the right side equals } M(\theta_0)-o_P(1)}$.

It follows that $$M_n(\hat{\theta}_n)\geq M(\theta_0)-o_P(1)$$

I don't really understand what's going one, I get that the right hand side should converge to something but that does not mean that it equals that something.

Because, assume we are taking limits, why are we allowed to take the limit just on the right side leaving the inequality unaffected? Furthermore if we are taking limits shouldn't the small oh $o_P(1)$ vanish?

Am I missing something here?

Answer 1

The assumption implies $|M_n(\theta_0)-M(\theta_0)|=o_P(1)$ (it's actually stronger than that). So the difference between $M_n(\theta_0)$ and $M(\theta_0)$ can be "eaten" by the $o_P(1)$, and in this sense $M_n(\theta_0)-o_P(1)$ and $M(\theta_0)-o_P(1)$ are the same, despite $M_n(\theta_0)$ being potentially different from $M(\theta_0)$ for all $n$.

The weird thing about equality involving things such as $o_P(1)$ is that $o_P(1)$ is a term that fulfills certain conditions without being fixed, so two different things can be equally and correctly called $o_P(1)$ without actually being precisely equal. $M_n(\theta_0)-o_P(1)$ and $M(\theta_0)-o_P(1)$ are equal in the sense that they are terms fulfilling exactly the same conditions as specified by the $o_P(1)$. Everything that "equals" $M_n(\theta_0)-o_P(1)$ also "equals" $M(\theta_0)-o_P(1)$, so they are "the same". (This is no longer true if you remove the $o_P(1)$, which is there to make this way of writing possible.)