# convergence rate of sample covariance matrix

I have a question about deriving the rate of convergence of sample covariance matrix. For the sake of simplicity, we can assume that our sample $$\{ X_i\}_{i=1}^{n}$$ is i.i.d. (I known we can relax this assumption). I have found that many papers mentioned that it is a $$\sqrt{n}-$$consistent estimator of true covariance matrix. My questions are

i) Is the consistency result obtained under the Frobenius norm? I have found that many matrix consistency used this norm;

ii) How to prove that $$\hat{\Sigma} = \frac{1}{n-1} \sum_{i=1}^{n} (X_i - \bar{X})(X_i-\bar{X})'= O_p(\frac{1}{\sqrt{n}})$$.

I have learned advanced probability theory, mathematical statistics, and so on. However, I am not familiar with the classical methods for proving large sample results and determining the rate of convergence of some estimators in some classical statistical models, such as linear model, factor model and so on, which caused me a lot of trouble to read some cutting-edge papers.

Any related posts and materials will be appreciated. Thank you in advance!

• Hi: I don't remember if it's explained as clearly as Thomas's explanation below, but Halbert White's "asymptotic theory for econometricians" has some of that material in it. If the relevant material that you're looking for is not in there, then I'm sure that the references in there will be helpful. Mar 31, 2021 at 3:22
• @mlofton Thank you, and I'll read this book. Mar 31, 2021 at 3:56

Start with $$S=\frac{1}{n}\sum_i (X_i-\mu)(X_i-\mu)'$$

This is an iid mean. Assuming the summands $$(X_i-\mu)(X_i-\mu)'$$ have two finite moments, the central limit theorem says that $$\sqrt{n}(S-E[S])\stackrel{d}{\to} N(0, V)$$ where $$V=\mathrm{var}[(X_i-\mu)(X_i-\mu)']$$. This means $$S-E[S]=O_p(1/\sqrt{n})$$.

This is for the Frobenius norm, but it's also true for the spectral norm.

Now, $$n/(n-1)=1+O(1/n)$$, so $$\tilde S=\frac{1}{n-1}\sum_i (X_i-\mu)(X_i-\mu)'$$ also converges at $$\sqrt{n}$$ rate, and clearly $$\bar X$$ converges to $$\mu$$ at $$\sqrt{n}$$ rate, so $$\hat \Sigma$$ is also $$\sqrt{n}$$-consistent.

So, basically, it's true for the same reasons that everything else is $$\sqrt{n}$$-consistent.

• Thank you. Your helpful answer provides the basic framework for solving most questions. Further, do you know how to get the rate of convergence in the non-i.i.d case? By some theorems similar to central limit theorem or some probability inequalities? Mar 31, 2021 at 4:05
• Yes, there are central limit theorems for many kinds of non-iid data, but they can be hard to find so it's sometimes easier to work out the rate at which the variance of the variance decreases with increasing $n$ and then just use Chebyshev's inequality to show convergence in probability at the square root of that rate. Mar 31, 2021 at 5:30
• What is $p$ in your answer? i.e. $O_p$ Aug 4 at 5:35
• I'm confused by what $S - E[S] \rightarrow N$ means. If I understand correctly, $S - E[S]$ is a matrix. What does it mean to say that this matrix follows a normal distribution; does this mean a matrix normal distribution? Aug 4 at 5:45
• Also, what is the variance of a matrix i.e $(X_i - \mu)(X_i - \mu)^{\prime}$? Aug 4 at 5:45