Proof of asymptotic variance How do you prove that
$X_n - E[X_n] = O_p(\sqrt{Var(X_n)})$
It's used in my textbook and I don't know where they get it from.
 A: For completion, I will provide an answer I found from Theorem 14.4-1 in Bishop et al. Understanding this helped me, and I wish to share it with others on this forum.
It goes as follows:
Step 1: Definition of $O_p(1)$
$X_n = O_p(1)$ if for every $\eta >0$ there exist a constant $K(\eta)$ and an integer $n(\eta)$ such that if $n\geq n(\eta)$ then $$P(|X_n|\leq K(\eta)) \geq 1-\eta$$.
Also $X_n = O_p(b_n)$ if $X_n / b_n = O_p(1)$, or equivalently $X_n = b_nO_p(1)$. It is sometimes useful to say that $X_n$ is bounded in probability.
Step 2: Tchebychev's inequality
If $X$ is a random variable with mean $\mu$ and variance $\sigma^2<\infty$ then $$P(|X-\mu|\leq h\sigma)\geq 1-h^{-2}$$
Step 3: the proof
In step 2, set $h=\eta^{-1/2}$ for any $0<\eta<1$ and apply it to $X_n$ we get
$$P(\dfrac{|X_n - E(X_n)|}{\sqrt{Var(X_n)}} < \eta^{-1/2}) \geq 1-\eta $$.
This holds for $n=1,2,3...$.  Setting $K(\eta) =\eta^{-1/2}$, we apply the definition in Step 1 and conclude that $$(X_n - E(X_n))/\sqrt{Var(X_n)} = O_p(1)$$.
Hence  $$X_n - E(X_n) = O_p(\sqrt{Var(X_n)})$$ as required.
