Suppose $E[( \theta-\hat{\theta}_n)^2] = O(n^{-1/2})$. Show that $\frac{1}{n}\sum_{i=1}^n (\theta-\hat{\theta}_n)^2 = O_p(n^{-1/2}).$ Assume that $E[( \theta-\hat{\theta}_n)^2] = O(n^{-1/2})$. How can I show that $\frac{1}{n}\sum_{i=1}^n (\theta-\hat{\theta}_n)^2 = O_p(n^{-1/2})?$
What I'm trying to ask is: if the expected value of some quantity is $O(n^{-1/2})$, does that imply the sample mean of that quantity is also $O_p(n^{-1/2})$?
 A: You want to show that for any $\varepsilon>0$, there exist two positive reals $M$ and $N$ such that
$$\mathbb P\left(\frac{1}{\sqrt n}\sum_{i=1}^n (\theta-\hat{\theta}_n)^2 > M\right)\le\varepsilon\ \ \forall n\ge N $$
Let $\varepsilon>0$. For any $n\ge1$ and $M>0$ you have, by Markov inequality, that
$$\begin{align}\mathbb P\left(\frac{1}{\sqrt n}\sum_{i=1}^n (\theta-\hat{\theta}_n)^2 > M\right)&\le\frac 1 M\mathbb E\left[\frac{1}{\sqrt n}\sum_{i=1}^n (\theta-\hat{\theta}_n)^2 \right]\\
&\le\frac{1}{M\sqrt n}\mathbb E\left[\sum_{i=1}^n (\theta-\hat{\theta}_n)^2 \right]\\
&\le\frac{\sqrt n}{M}\mathbb E\left[(\theta-\hat{\theta}_n)^2 \right]\\\end{align} $$
But, because you assume that $E[( \theta-\hat{\theta}_n)^2] = O(n^{-1/2})$ (it's a deterministic sequence so I assume the big O is the usual one, otherwise it makes no sense), you know that for any positive $M'$, there exists $N'$ such that, for all $n\ge N'$ :
$$\sqrt nE[( \theta-\hat{\theta}_n)^2] \le M' $$
Now take $M'=M\varepsilon$ and you can conclude that the desired inequality holds for all $n\ge N'$.
[Note that I assumed that the $\theta$ in the sum were supposed to be $n$ i.i.d. realizations of $\theta$ called $\theta_i$.]
