While I think it is reasonable, I cannot show this result.

Suppose $\hat{\theta}$ is an estimator of $\theta$ and $P(\hat{\theta}\neq \theta) \rightarrow 0$ as the sample size increases, that is, $P(\hat{\theta}\neq \theta)=o(1)$. Then I can write $\hat{\theta}=\theta+o_p(1)$.

How can I show this?

*Note the difference between, $o(1)$ and $o_p(1)$. While "little oh" deal with sequences of numbers, the "little oh in probability" is related with convergence in probability (to zero).

*The only similar result that I know is $\hat{\theta}=E(\hat{\theta})+(Var(\hat{\theta}))^{1/2} O_p(1)$. In my case, $\hat{\theta}=\min_{\theta_1}\{ RSS(\theta_1)+\lambda \theta_1\}$

  • 2
    $\begingroup$ I'm actually not aware of the difference between $o(1)$ and $o_p(1)$. Does it have to do with strict boundedness vs. boundedness in probability? $\endgroup$
    – AdamO
    Jun 3, 2019 at 15:50
  • $\begingroup$ @AdamO ok. I Updated my question! $\endgroup$ Jun 3, 2019 at 15:57

1 Answer 1


That $\hat{\theta} = \theta + o_{p}(1)$ is a restatement of $\hat{\theta} \to_{p} \theta$. (Aware that saying a sequence $(x_{n})$ converges to some $x$ is equivalent to saying $x_{n} = x + o(1)$.) The latter statement holds iff $\mathbb{P}(|\hat{\theta} - \theta| \geq \epsilon) \to 0$ for every $\epsilon > 0$, by definition. Since $|\hat{\theta} - \theta| \geq \epsilon$ implies $\hat{\theta} \neq \theta$, we have $$ \mathbb{P}(|\hat{\theta} - \theta| \geq \epsilon) \leq \mathbb{P}(\hat{\theta} \neq \theta) \to 0 $$ by assumption.

  • $\begingroup$ Nice. I believe the key point is the last inequality. For a given $\epsilon$ , in fact, the inverse image $\{\hat{\theta} \neq \theta\}$ contains $\{\mid \hat{\theta}-\theta\mid\}$. Using the monotonicity of the measure, the result follows. But I also need an argument about the equivalence that you claimed in the first sentence. $\endgroup$ Jun 3, 2019 at 16:36
  • $\begingroup$ Ok, I got it now. hehe $\endgroup$ Jun 3, 2019 at 16:42
  • $\begingroup$ @Danmat, The first claim is by convention. The point seems to be the conclusion of a convergence in probability. Unless there is another meaning assigned to the statement pertaining to $o_{p}(1)$, the first claim is not much crucial. $\endgroup$
    – Yes
    Jun 3, 2019 at 16:59
  • $\begingroup$ @GaryMoore Yes, I thought about the first calim, and concluded that the equivalence follows directly by the definition of convergence in probability and the definition of $o_p$. Thanks for the answer! $\endgroup$ Jun 3, 2019 at 17:08
  • $\begingroup$ @Danmat, you are welcome. I guess it was perhaps the casual "so" that confused you. $\endgroup$
    – Yes
    Jun 3, 2019 at 17:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.