# $P(\hat{\theta}\neq \theta) \rightarrow 0$ as the sample size increases implies $\hat{\theta}= \theta+o_p(1)$?

While I think it is reasonable, I cannot show this result.

Suppose $$\hat{\theta}$$ is an estimator of $$\theta$$ and $$P(\hat{\theta}\neq \theta) \rightarrow 0$$ as the sample size increases, that is, $$P(\hat{\theta}\neq \theta)=o(1)$$. Then I can write $$\hat{\theta}=\theta+o_p(1)$$.

How can I show this?

*Note the difference between, $$o(1)$$ and $$o_p(1)$$. While "little oh" deal with sequences of numbers, the "little oh in probability" is related with convergence in probability (to zero).

*The only similar result that I know is $$\hat{\theta}=E(\hat{\theta})+(Var(\hat{\theta}))^{1/2} O_p(1)$$. In my case, $$\hat{\theta}=\min_{\theta_1}\{ RSS(\theta_1)+\lambda \theta_1\}$$

• I'm actually not aware of the difference between $o(1)$ and $o_p(1)$. Does it have to do with strict boundedness vs. boundedness in probability? Jun 3, 2019 at 15:50
• @AdamO ok. I Updated my question! Jun 3, 2019 at 15:57

That $$\hat{\theta} = \theta + o_{p}(1)$$ is a restatement of $$\hat{\theta} \to_{p} \theta$$. (Aware that saying a sequence $$(x_{n})$$ converges to some $$x$$ is equivalent to saying $$x_{n} = x + o(1)$$.) The latter statement holds iff $$\mathbb{P}(|\hat{\theta} - \theta| \geq \epsilon) \to 0$$ for every $$\epsilon > 0$$, by definition. Since $$|\hat{\theta} - \theta| \geq \epsilon$$ implies $$\hat{\theta} \neq \theta$$, we have $$\mathbb{P}(|\hat{\theta} - \theta| \geq \epsilon) \leq \mathbb{P}(\hat{\theta} \neq \theta) \to 0$$ by assumption.
• Nice. I believe the key point is the last inequality. For a given $\epsilon$ , in fact, the inverse image $\{\hat{\theta} \neq \theta\}$ contains $\{\mid \hat{\theta}-\theta\mid\}$. Using the monotonicity of the measure, the result follows. But I also need an argument about the equivalence that you claimed in the first sentence. Jun 3, 2019 at 16:36
• @Danmat, The first claim is by convention. The point seems to be the conclusion of a convergence in probability. Unless there is another meaning assigned to the statement pertaining to $o_{p}(1)$, the first claim is not much crucial.
• @GaryMoore Yes, I thought about the first calim, and concluded that the equivalence follows directly by the definition of convergence in probability and the definition of $o_p$. Thanks for the answer! Jun 3, 2019 at 17:08