Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

How do you prove that

$X_n - E[X_n] = O_p(\sqrt{Var(X_n)})$

It's used in my textbook and I don't know where they get it from.

share|improve this question
Are there any assumptions stated regarding the sequence of random variables $\{X_n\}$? Are they identically distributed? Independently? – Alecos Papadopoulos Aug 24 '14 at 11:26
The joint law of the $X_n$ is not important here. The result is a straightforward application of Chebyshev's inequality (and the definition of big O). – Stéphane Laurent Aug 24 '14 at 11:58
@StéphaneLaurent My question to the OP intended to find out the framework into which the textbook in question presents the equality in question -not the actual conditions needed (or needed not) for the equality to hold. – Alecos Papadopoulos Aug 24 '14 at 12:12
@AlecosPapadopoulos My first sentence was not intended to contradict you, but only to set the framework of my second sentence. – Stéphane Laurent Aug 24 '14 at 12:21
up vote 6 down vote accepted

For completion, I will provide an answer I found from Theorem 14.4-1 in Bishop et al. Understanding this helped me, and I wish to share it with others on this forum.

It goes as follows:

Step 1: Definition of $O_p(1)$

$X_n = O_p(1)$ if for every $\eta >0$ there exist a constant $K(\eta)$ and an integer $n(\eta)$ such that if $n\geq n(\eta)$ then $$P(|X_n|\leq K(\eta)) \geq 1-\eta$$.

Also $X_n = O_p(b_n)$ if $X_n / b_n = O_p(1)$, or equivalently $X_n = b_nO_p(1)$. It is sometimes useful to say that $X_n$ is bounded in probability.

Step 2: Tchebychev's inequality

If $X$ is a random variable with mean $\mu$ and variance $\sigma^2<\infty$ then $$P(|X-\mu|\leq h\sigma)\geq 1-h^{-2}$$

Step 3: the proof

In step 2, set $h=\eta^{-1/2}$ for any $0<\eta<1$ and apply it to $X_n$ we get $$P(\dfrac{|X_n - E(X_n)|}{\sqrt{Var(X_n)}} < \eta^{-1/2}) \geq 1-\eta $$.

This holds for $n=1,2,3...$. Setting $K(\eta) =\eta^{-1/2}$, we apply the definition in Step 1 and conclude that $$(X_n - E(X_n))/\sqrt{Var(X_n)} = O_p(1)$$.

Hence $$X_n - E(X_n) = O_p(\sqrt{Var(X_n)})$$ as required.

share|improve this answer
+1 for posting the answer. – Alecos Papadopoulos Aug 24 '14 at 21:35

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.