Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Consider $\{X_i\}$ i.i.d. sample of multivariate (in $\mathbb{R}^J$) with mean $\mu$ and variance $\Sigma$. I've been asked to show that $\hat{\Sigma}\equiv\frac{1}{n}\sum_{i=1}^n (X_i-\bar{X}_n) (X_i-\bar{X}_n)'$ converges almost surely to $\Sigma$.
I have no idea how to star that, besides using Central Limit Theorem on $\bar{X}_n$, and I don't know what to do with that. Can anyone give me some hints and ideas on how to proceed? Thank you for your help.
The idea is to open the sample variance: $$ \hat{\Sigma}=\frac{1}{n}\sum_{i=1}^nX_iX_i'-\bar{X}_n\bar{X}_n' $$ and notice that by the Strong Law of Large Numbers and the Continuous Mapping Theorem (actually a corollary of it, probably known as "Algebra of Stochastic Convergence"): $$ \bar{X}_n\overset{a.s.}{\to}\mu\implies\bar{X}_n\bar{X}_n'\overset{a.s.}{\to}\mu\mu'\\ \frac{1}{n}\sum_{i=1}^nX_iX_i'\overset{a.s.}{\to}E(X_iX_i') $$ where the convergence on the second line happens because the variance exists, so that the expectation of the product $X_iX_i'$ is well-defined. And using Continuous Mapping theorem again we can subtract both and maintain the convergence: $$ \hat{\Sigma}\overset{a.s.}{\to}\Sigma $$
