# Consistency of M-estimator?

The reported results can be found in van der Vaart's "Asymptotic Statistics".

I am having some difficulties to understand the logic behind the following proof provided by the author:

Theorem $$5.7.$$ Let $$M_n$$ be random functions and let $$M$$ be a fixed function of $$\theta$$ such that $$\forall \epsilon>0$$

$$\sup_{\theta} \vert M_n(\theta)-M(\theta)\vert\to 0,$$ $$\sup_{\theta:d(\theta,\theta_0)\geq \epsilon} M(\theta) Then any sequence of estimators $$\hat{\theta}_n$$ with $$M_n(\hat{\theta}_n)\geq M_n(\theta_0)-o_P(1)$$ converges in probability to $$\theta_0$$.

(all functions live in a space topologized by convergence in probability so all limit's should be understood as limit in probability).

Proof:

Assume $$\hat{\theta}_n$$ satisfies the conditions, so we have $$M_n(\hat{\theta}_n)\geq M_n(\theta_0)-o_P(1)$$

Because uniform convergence of $$M_n$$ to $$M$$ implies convergence of $$M_n(\theta_0)$$ to $$M(\theta_0)$$ $$\color{red}{\text{ the right side equals } M(\theta_0)-o_P(1)}$$.

It follows that $$M_n(\hat{\theta}_n)\geq M(\theta_0)-o_P(1)$$

I don't really understand what's going one, I get that the right hand side should converge to something but that does not mean that it equals that something.

Because, assume we are taking limits, why are we allowed to take the limit just on the right side leaving the inequality unaffected? Furthermore if we are taking limits shouldn't the small oh $$o_P(1)$$ vanish?

Am I missing something here?

• This doesn't seem to be on M-estimators, rather on general estimators just involving functions that are called $M$. So the title is probably misleading. There are also two rather obvious typos in the Theorem (the "$>$" after the sup, and "convergence in probability") that I wasn't allowed to edit because... "edits have to change at least 6 characters"!?? I think it helps reading to correst these. – Lewian Feb 14 at 15:47
• Probably "the right side equals $M(\theta_0)-o_P(1)$" should be read "if the right side is replaced by $M(\theta_0)-o_P(1)$, the inequality still holds" and then it is clearer? – Lewian Feb 14 at 15:50
• @Lewian hey thanks for pointing out the typos! I don't see where I should correct the "convergence in probability", do you mind telling me to which part do you refer? – RScrlli Feb 14 at 15:54
• With respect to the title, actually the author uses this notation to speak of M-estimators, even though the theorem indeed seems quite general – RScrlli Feb 14 at 15:56
• "converges is probability" > "converges in probability" – Lewian Feb 14 at 15:58

The assumption implies $$|M_n(\theta_0)-M(\theta_0)|=o_P(1)$$ (it's actually stronger than that). So the difference between $$M_n(\theta_0)$$ and $$M(\theta_0)$$ can be "eaten" by the $$o_P(1)$$, and in this sense $$M_n(\theta_0)-o_P(1)$$ and $$M(\theta_0)-o_P(1)$$ are the same, despite $$M_n(\theta_0)$$ being potentially different from $$M(\theta_0)$$ for all $$n$$.
The weird thing about equality involving things such as $$o_P(1)$$ is that $$o_P(1)$$ is a term that fulfills certain conditions without being fixed, so two different things can be equally and correctly called $$o_P(1)$$ without actually being precisely equal. $$M_n(\theta_0)-o_P(1)$$ and $$M(\theta_0)-o_P(1)$$ are equal in the sense that they are terms fulfilling exactly the same conditions as specified by the $$o_P(1)$$. Everything that "equals" $$M_n(\theta_0)-o_P(1)$$ also "equals" $$M(\theta_0)-o_P(1)$$, so they are "the same". (This is no longer true if you remove the $$o_P(1)$$, which is there to make this way of writing possible.)